[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from AutoRebalancerTest
[ RUN ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251023 02:07:56.370288 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:38547,127.15.161.253:39547,127.15.161.252:38849
I20251023 02:07:56.372116 16007 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251023 02:07:56.373274 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:56.383513 16013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.383524 16012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.383527 16015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.384196 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:56.384979 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:56.385131 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:56.385195 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185276385180 us; error 0 us; skew 500 ppm
I20251023 02:07:56.385471 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:56.387856 16007 webserver.cc:492] Webserver started at http://127.15.161.254:39833/ using document root <none> and password file <none>
I20251023 02:07:56.388520 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:56.388613 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:56.388924 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:56.391815 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "2af3410a78774d75b6665728ecb20530"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.397100 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20251023 02:07:56.400061 16020 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.401417 16007 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20251023 02:07:56.401577 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "2af3410a78774d75b6665728ecb20530"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.401746 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:56.425513 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:56.426455 16007 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251023 02:07:56.426718 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:56.445797 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:38547
I20251023 02:07:56.445827 16071 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:38547 every 8 connection(s)
I20251023 02:07:56.448027 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:56.450321 16074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.450500 16075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.451560 16072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251023 02:07:56.451694 16077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.451804 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:56.452162 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:56.452239 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:56.452306 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185276452273 us; error 0 us; skew 500 ppm
I20251023 02:07:56.452450 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:56.453732 16007 webserver.cc:492] Webserver started at http://127.15.161.253:36677/ using document root <none> and password file <none>
I20251023 02:07:56.453984 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:56.454088 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:56.454247 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:56.455040 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root/instance:
uuid: "15bb4c0004234d44a4e232cc464ffad3"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.458211 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251023 02:07:56.460403 16082 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.460839 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 02:07:56.460964 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root
uuid: "15bb4c0004234d44a4e232cc464ffad3"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.461095 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:56.464125 16072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.477559 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:56.478075 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:56.485519 16072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
W20251023 02:07:56.487123 16022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.253:39547: connect: Connection refused (error 111)
W20251023 02:07:56.489167 16072 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.253:39547: Network error: Client connection negotiation failed: client connection to 127.15.161.253:39547: connect: Connection refused (error 111)
I20251023 02:07:56.494176 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.253:39547
I20251023 02:07:56.494236 16136 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.253:39547 every 8 connection(s)
I20251023 02:07:56.495630 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:07:56.495697 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251023 02:07:56.498914 16139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.499572 16140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.499676 16137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } has no permanent_uuid. Determining permanent_uuid...
W20251023 02:07:56.500373 16142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.500571 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:56.500938 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:56.501000 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:56.501052 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185276501020 us; error 0 us; skew 500 ppm
I20251023 02:07:56.501186 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:56.502467 16007 webserver.cc:492] Webserver started at http://127.15.161.252:41387/ using document root <none> and password file <none>
I20251023 02:07:56.502686 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:56.502777 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:56.502902 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:56.503489 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root/instance:
uuid: "4be0e458870b43d394b91214fdbc4ab2"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.506328 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20251023 02:07:56.507794 16137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.508342 16148 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.508770 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:07:56.508885 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root
uuid: "4be0e458870b43d394b91214fdbc4ab2"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.509073 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:56.514866 16137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.516758 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20251023 02:07:56.517086 16137 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.252:38849: Network error: Client connection negotiation failed: client connection to 127.15.161.252:38849: connect: Connection refused (error 111)
I20251023 02:07:56.517376 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:56.531387 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.252:38849
I20251023 02:07:56.531445 16201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.252:38849 every 8 connection(s)
I20251023 02:07:56.532368 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:07:56.532764 16202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:07:56.534950 16202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.538458 16072 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } attempt: 1
I20251023 02:07:56.541167 16202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.544358 16072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.547951 16202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:07:56.555660 16072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: Bootstrap starting.
I20251023 02:07:56.556689 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: Bootstrap starting.
I20251023 02:07:56.560433 16202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:56.560438 16072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:56.562088 16202 log.cc:826] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: Log is configured to *not* fsync() on all Append() calls
I20251023 02:07:56.562088 16072 log.cc:826] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: Log is configured to *not* fsync() on all Append() calls
I20251023 02:07:56.565032 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: No bootstrap required, opened a new log
I20251023 02:07:56.565037 16072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: No bootstrap required, opened a new log
I20251023 02:07:56.569383 16137 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } attempt: 1
I20251023 02:07:56.571131 16072 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.571132 16202 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.571540 16202 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:56.571542 16072 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:56.571718 16202 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be0e458870b43d394b91214fdbc4ab2, State: Initialized, Role: FOLLOWER
I20251023 02:07:56.571712 16072 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af3410a78774d75b6665728ecb20530, State: Initialized, Role: FOLLOWER
I20251023 02:07:56.572515 16202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.572516 16072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.573537 16210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.573922 16210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.574911 16211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.575184 16072 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:07:56.575184 16202 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:07:56.575258 16211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.579771 16137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3: Bootstrap starting.
I20251023 02:07:56.582266 16137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:56.583216 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251023 02:07:56.588341 16229 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:07:56.588471 16229 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 02:07:56.588477 16137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3: No bootstrap required, opened a new log
W20251023 02:07:56.588949 16234 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:07:56.589030 16234 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 02:07:56.589345 16137 raft_consensus.cc:359] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.589568 16137 raft_consensus.cc:385] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:56.589650 16137 raft_consensus.cc:740] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15bb4c0004234d44a4e232cc464ffad3, State: Initialized, Role: FOLLOWER
I20251023 02:07:56.589893 16137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.590579 16236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.590792 16236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.590884 16137 sys_catalog.cc:565] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:07:56.595467 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251023 02:07:56.596529 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:56.596956 16247 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:07:56.597061 16247 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251023 02:07:56.600253 16248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.600273 16249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.603544 16251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.603855 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:56.604130 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:56.604211 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:56.604277 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185276604249 us; error 0 us; skew 500 ppm
I20251023 02:07:56.604437 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:56.605749 16007 webserver.cc:492] Webserver started at http://127.15.161.193:46277/ using document root <none> and password file <none>
I20251023 02:07:56.606019 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:56.606104 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:56.606312 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:56.607026 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "6f6e4dd23eb341bc82aa9d24a634ba78"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.609907 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251023 02:07:56.611624 16256 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.611994 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:07:56.612083 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "6f6e4dd23eb341bc82aa9d24a634ba78"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.612202 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:56.619616 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:56.620247 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:56.621039 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:07:56.622488 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:07:56.622588 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.622705 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:07:56.622785 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.633311 16236 raft_consensus.cc:493] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:07:56.633675 16236 raft_consensus.cc:515] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.637089 16236 leader_election.cc:290] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2af3410a78774d75b6665728ecb20530 (127.15.161.254:38547), 4be0e458870b43d394b91214fdbc4ab2 (127.15.161.252:38849)
I20251023 02:07:56.637053 16047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15bb4c0004234d44a4e232cc464ffad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af3410a78774d75b6665728ecb20530" is_pre_election: true
I20251023 02:07:56.637487 16047 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15bb4c0004234d44a4e232cc464ffad3 in term 0.
I20251023 02:07:56.637940 16177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15bb4c0004234d44a4e232cc464ffad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be0e458870b43d394b91214fdbc4ab2" is_pre_election: true
I20251023 02:07:56.638216 16177 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15bb4c0004234d44a4e232cc464ffad3 in term 0.
I20251023 02:07:56.638808 16086 leader_election.cc:304] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15bb4c0004234d44a4e232cc464ffad3, 2af3410a78774d75b6665728ecb20530; no voters:
I20251023 02:07:56.639477 16236 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:07:56.639605 16236 raft_consensus.cc:493] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:07:56.639703 16236 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:56.642755 16236 raft_consensus.cc:515] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.643424 16236 leader_election.cc:290] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [CANDIDATE]: Term 1 election: Requested vote from peers 2af3410a78774d75b6665728ecb20530 (127.15.161.254:38547), 4be0e458870b43d394b91214fdbc4ab2 (127.15.161.252:38849)
I20251023 02:07:56.643879 16047 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15bb4c0004234d44a4e232cc464ffad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af3410a78774d75b6665728ecb20530"
I20251023 02:07:56.644054 16047 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:56.644258 16177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15bb4c0004234d44a4e232cc464ffad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be0e458870b43d394b91214fdbc4ab2"
I20251023 02:07:56.644477 16177 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:56.647254 16047 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15bb4c0004234d44a4e232cc464ffad3 in term 1.
I20251023 02:07:56.647812 16086 leader_election.cc:304] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15bb4c0004234d44a4e232cc464ffad3, 2af3410a78774d75b6665728ecb20530; no voters:
I20251023 02:07:56.648177 16236 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:07:56.648402 16177 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15bb4c0004234d44a4e232cc464ffad3 in term 1.
I20251023 02:07:56.649297 16236 raft_consensus.cc:697] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 1 LEADER]: Becoming Leader. State: Replica: 15bb4c0004234d44a4e232cc464ffad3, State: Running, Role: LEADER
I20251023 02:07:56.649991 16236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } }
I20251023 02:07:56.657040 16294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15bb4c0004234d44a4e232cc464ffad3. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.657444 16294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: This master's current role is: LEADER
I20251023 02:07:56.672448 16308 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:07:56.674830 16308 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20251023 02:07:56.678689 16026 tablet.cc:2378] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 02:07:56.691210 16177 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 1 FOLLOWER]: Refusing update from remote peer 15bb4c0004234d44a4e232cc464ffad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:07:56.692040 16294 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:07:56.699036 16047 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 1 FOLLOWER]: Refusing update from remote peer 15bb4c0004234d44a4e232cc464ffad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:07:56.699702 16294 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:07:56.717249 16211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15bb4c0004234d44a4e232cc464ffad3. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.717520 16211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.721423 16294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.721768 16294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: This master's current role is: LEADER
I20251023 02:07:56.723255 16308 catalog_manager.cc:1357] Generated new cluster ID: d9974282b2e040168d2358b7408ebdcf
I20251023 02:07:56.723342 16308 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:07:56.726994 16211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.727171 16211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.730350 16315 mvcc.cc:204] Tried to move back new op lower bound from 7213814893314527232 to 7213814893164535808. Current Snapshot: MvccSnapshot[applied={T|T < 7213814893314527232 or (T in {7213814893314527232})}]
I20251023 02:07:56.737613 16210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15bb4c0004234d44a4e232cc464ffad3. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.737921 16210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.739516 16294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.739718 16294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [sys.catalog]: This master's current role is: LEADER
I20251023 02:07:56.747938 16210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15bb4c0004234d44a4e232cc464ffad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af3410a78774d75b6665728ecb20530" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 38547 } } peers { permanent_uuid: "15bb4c0004234d44a4e232cc464ffad3" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 39547 } } peers { permanent_uuid: "4be0e458870b43d394b91214fdbc4ab2" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 38849 } } }
I20251023 02:07:56.748180 16210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:07:56.753669 16308 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:07:56.755460 16308 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:07:56.766498 16329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:40877 every 8 connection(s)
I20251023 02:07:56.766495 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:40877
I20251023 02:07:56.773746 16308 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3: Generated new TSK 0
I20251023 02:07:56.775939 16308 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:07:56.782702 16330 heartbeater.cc:344] Connected to a master server at 127.15.161.252:38849
I20251023 02:07:56.782982 16330 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.784142 16330 heartbeater.cc:507] Master 127.15.161.252:38849 requested a full tablet report, sending...
I20251023 02:07:56.787371 16167 ts_manager.cc:194] Registered new tserver with Master: 6f6e4dd23eb341bc82aa9d24a634ba78 (127.15.161.193:40877)
I20251023 02:07:56.795815 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:07:56.797124 16331 heartbeater.cc:344] Connected to a master server at 127.15.161.254:38547
I20251023 02:07:56.797283 16331 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.797868 16331 heartbeater.cc:507] Master 127.15.161.254:38547 requested a full tablet report, sending...
I20251023 02:07:56.799162 16037 ts_manager.cc:194] Registered new tserver with Master: 6f6e4dd23eb341bc82aa9d24a634ba78 (127.15.161.193:40877)
I20251023 02:07:56.804494 16332 heartbeater.cc:344] Connected to a master server at 127.15.161.253:39547
W20251023 02:07:56.804575 16336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.804733 16332 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.805239 16332 heartbeater.cc:507] Master 127.15.161.253:39547 requested a full tablet report, sending...
I20251023 02:07:56.806465 16101 ts_manager.cc:194] Registered new tserver with Master: 6f6e4dd23eb341bc82aa9d24a634ba78 (127.15.161.193:40877)
I20251023 02:07:56.809564 16101 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45208
W20251023 02:07:56.812139 16337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:56.812978 16339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.814811 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:56.815133 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:56.815207 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:56.815290 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185276815246 us; error 0 us; skew 500 ppm
I20251023 02:07:56.815459 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:56.816815 16007 webserver.cc:492] Webserver started at http://127.15.161.194:40967/ using document root <none> and password file <none>
I20251023 02:07:56.817014 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:56.817107 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:56.817236 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:56.817893 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "0c761e8a5ed8401e8907a79efd8277f3"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.820339 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 02:07:56.822002 16344 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.822453 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:07:56.822594 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "0c761e8a5ed8401e8907a79efd8277f3"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.822757 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:56.842255 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:56.842991 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:56.843713 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:07:56.844779 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:07:56.844861 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.844974 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:07:56.845055 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.862689 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:40971
I20251023 02:07:56.862708 16406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:40971 every 8 connection(s)
I20251023 02:07:56.900061 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:07:56.900614 16408 heartbeater.cc:344] Connected to a master server at 127.15.161.254:38547
I20251023 02:07:56.900763 16408 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.901288 16408 heartbeater.cc:507] Master 127.15.161.254:38547 requested a full tablet report, sending...
I20251023 02:07:56.902612 16037 ts_manager.cc:194] Registered new tserver with Master: 0c761e8a5ed8401e8907a79efd8277f3 (127.15.161.194:40971)
W20251023 02:07:56.906771 16414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.907615 16407 heartbeater.cc:344] Connected to a master server at 127.15.161.252:38849
I20251023 02:07:56.907745 16407 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.908217 16407 heartbeater.cc:507] Master 127.15.161.252:38849 requested a full tablet report, sending...
I20251023 02:07:56.909438 16167 ts_manager.cc:194] Registered new tserver with Master: 0c761e8a5ed8401e8907a79efd8277f3 (127.15.161.194:40971)
I20251023 02:07:56.909873 16409 heartbeater.cc:344] Connected to a master server at 127.15.161.253:39547
I20251023 02:07:56.910046 16409 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.910742 16409 heartbeater.cc:507] Master 127.15.161.253:39547 requested a full tablet report, sending...
I20251023 02:07:56.911880 16101 ts_manager.cc:194] Registered new tserver with Master: 0c761e8a5ed8401e8907a79efd8277f3 (127.15.161.194:40971)
I20251023 02:07:56.913280 16101 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45222
W20251023 02:07:56.914244 16415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.916747 16007 server_base.cc:1047] running on GCE node
W20251023 02:07:56.916929 16417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:56.917299 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:56.917392 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:56.917450 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185276917429 us; error 0 us; skew 500 ppm
I20251023 02:07:56.917619 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:56.918922 16007 webserver.cc:492] Webserver started at http://127.15.161.195:43003/ using document root <none> and password file <none>
I20251023 02:07:56.919165 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:56.919250 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:56.919409 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:56.920181 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "0aefe06419b84268854856fb2256f037"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.923287 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251023 02:07:56.926575 16422 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.927285 16007 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20251023 02:07:56.927405 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "0aefe06419b84268854856fb2256f037"
format_stamp: "Formatted at 2025-10-23 02:07:56 on dist-test-slave-bfbt"
I20251023 02:07:56.927567 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:56.953482 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:56.954331 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:56.955228 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:07:56.956439 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:07:56.956537 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.956634 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:07:56.956712 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:56.974880 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:33157
I20251023 02:07:56.974964 16484 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:33157 every 8 connection(s)
I20251023 02:07:56.992631 16485 heartbeater.cc:344] Connected to a master server at 127.15.161.252:38849
I20251023 02:07:56.992816 16485 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:56.993427 16485 heartbeater.cc:507] Master 127.15.161.252:38849 requested a full tablet report, sending...
I20251023 02:07:56.994542 16167 ts_manager.cc:194] Registered new tserver with Master: 0aefe06419b84268854856fb2256f037 (127.15.161.195:33157)
I20251023 02:07:57.009295 16486 heartbeater.cc:344] Connected to a master server at 127.15.161.254:38547
I20251023 02:07:57.009480 16486 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:57.010571 16486 heartbeater.cc:507] Master 127.15.161.254:38547 requested a full tablet report, sending...
I20251023 02:07:57.011793 16037 ts_manager.cc:194] Registered new tserver with Master: 0aefe06419b84268854856fb2256f037 (127.15.161.195:33157)
I20251023 02:07:57.012075 16487 heartbeater.cc:344] Connected to a master server at 127.15.161.253:39547
I20251023 02:07:57.012209 16487 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:57.012671 16487 heartbeater.cc:507] Master 127.15.161.253:39547 requested a full tablet report, sending...
I20251023 02:07:57.013393 16101 ts_manager.cc:194] Registered new tserver with Master: 0aefe06419b84268854856fb2256f037 (127.15.161.195:33157)
I20251023 02:07:57.013645 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013396974s
I20251023 02:07:57.014151 16101 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45232
I20251023 02:07:57.590302 16229 catalog_manager.cc:1269] Loaded cluster ID: d9974282b2e040168d2358b7408ebdcf
I20251023 02:07:57.590303 16234 catalog_manager.cc:1269] Loaded cluster ID: d9974282b2e040168d2358b7408ebdcf
I20251023 02:07:57.590436 16229 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: loading cluster ID for follower catalog manager: success
I20251023 02:07:57.590444 16234 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: loading cluster ID for follower catalog manager: success
I20251023 02:07:57.592496 16234 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: acquiring CA information for follower catalog manager: success
I20251023 02:07:57.592533 16229 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: acquiring CA information for follower catalog manager: success
I20251023 02:07:57.593791 16234 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 02:07:57.593796 16229 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 02:07:57.812001 16332 heartbeater.cc:499] Master 127.15.161.253:39547 was elected leader, sending a full tablet report...
I20251023 02:07:57.915223 16409 heartbeater.cc:499] Master 127.15.161.253:39547 was elected leader, sending a full tablet report...
I20251023 02:07:58.015224 16487 heartbeater.cc:499] Master 127.15.161.253:39547 was elected leader, sending a full tablet report...
I20251023 02:07:58.595479 16246 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:07:59.072747 16007 test_util.cc:276] Using random seed: -420390445
I20251023 02:07:59.107184 16101 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45234:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251023 02:07:59.151860 16283 tablet_service.cc:1505] Processing CreateTablet for tablet bf9536a59806474fa1e3e75cc97859e6 (DEFAULT_TABLE table=test-workload [id=1577d63702f6452eb70ae48bd53ff496]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:07:59.151916 16284 tablet_service.cc:1505] Processing CreateTablet for tablet cd6d425395924da1b159363891228c78 (DEFAULT_TABLE table=test-workload [id=1577d63702f6452eb70ae48bd53ff496]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:07:59.153025 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf9536a59806474fa1e3e75cc97859e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:07:59.153256 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd6d425395924da1b159363891228c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:07:59.156036 16372 tablet_service.cc:1505] Processing CreateTablet for tablet da79a38919894cfebbbb21b4f0373fb8 (DEFAULT_TABLE table=test-workload [id=1577d63702f6452eb70ae48bd53ff496]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:07:59.156577 16372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da79a38919894cfebbbb21b4f0373fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:07:59.160142 16520 tablet_bootstrap.cc:492] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78: Bootstrap starting.
I20251023 02:07:59.161682 16520 tablet_bootstrap.cc:654] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:59.162530 16521 tablet_bootstrap.cc:492] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3: Bootstrap starting.
I20251023 02:07:59.165278 16521 tablet_bootstrap.cc:654] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:59.167101 16450 tablet_service.cc:1505] Processing CreateTablet for tablet 2c55b3d4976a4d2c903405ab265bd87e (DEFAULT_TABLE table=test-workload [id=1577d63702f6452eb70ae48bd53ff496]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:07:59.167764 16450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c55b3d4976a4d2c903405ab265bd87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:07:59.170295 16521 tablet_bootstrap.cc:492] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3: No bootstrap required, opened a new log
I20251023 02:07:59.170473 16521 ts_tablet_manager.cc:1403] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20251023 02:07:59.170693 16520 tablet_bootstrap.cc:492] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78: No bootstrap required, opened a new log
I20251023 02:07:59.170840 16520 ts_tablet_manager.cc:1403] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20251023 02:07:59.171340 16521 raft_consensus.cc:359] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c761e8a5ed8401e8907a79efd8277f3" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40971 } }
I20251023 02:07:59.171412 16520 raft_consensus.cc:359] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.171576 16521 raft_consensus.cc:385] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:59.171594 16520 raft_consensus.cc:385] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:59.171756 16521 raft_consensus.cc:740] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c761e8a5ed8401e8907a79efd8277f3, State: Initialized, Role: FOLLOWER
I20251023 02:07:59.171788 16520 raft_consensus.cc:740] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6e4dd23eb341bc82aa9d24a634ba78, State: Initialized, Role: FOLLOWER
I20251023 02:07:59.172057 16520 consensus_queue.cc:260] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.172062 16521 consensus_queue.cc:260] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c761e8a5ed8401e8907a79efd8277f3" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40971 } }
I20251023 02:07:59.172289 16520 raft_consensus.cc:399] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:07:59.172313 16521 raft_consensus.cc:399] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:07:59.172430 16520 raft_consensus.cc:493] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:07:59.172433 16521 raft_consensus.cc:493] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:07:59.172603 16520 raft_consensus.cc:3060] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:59.172628 16521 raft_consensus.cc:3060] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:59.174192 16524 tablet_bootstrap.cc:492] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037: Bootstrap starting.
I20251023 02:07:59.175515 16520 raft_consensus.cc:515] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.175514 16521 raft_consensus.cc:515] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c761e8a5ed8401e8907a79efd8277f3" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40971 } }
I20251023 02:07:59.175849 16524 tablet_bootstrap.cc:654] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:59.175832 16521 leader_election.cc:304] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c761e8a5ed8401e8907a79efd8277f3; no voters:
I20251023 02:07:59.175856 16520 leader_election.cc:304] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f6e4dd23eb341bc82aa9d24a634ba78; no voters:
I20251023 02:07:59.176510 16521 leader_election.cc:290] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:07:59.176745 16525 raft_consensus.cc:2804] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:07:59.181262 16524 tablet_bootstrap.cc:492] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037: No bootstrap required, opened a new log
I20251023 02:07:59.181485 16524 ts_tablet_manager.cc:1403] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.004s
I20251023 02:07:59.182446 16524 raft_consensus.cc:359] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aefe06419b84268854856fb2256f037" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 33157 } }
I20251023 02:07:59.182689 16524 raft_consensus.cc:385] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:59.182816 16524 raft_consensus.cc:740] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aefe06419b84268854856fb2256f037, State: Initialized, Role: FOLLOWER
I20251023 02:07:59.183147 16524 consensus_queue.cc:260] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aefe06419b84268854856fb2256f037" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 33157 } }
I20251023 02:07:59.183363 16524 raft_consensus.cc:399] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:07:59.183459 16524 raft_consensus.cc:493] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:07:59.183584 16524 raft_consensus.cc:3060] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:59.187224 16524 raft_consensus.cc:515] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aefe06419b84268854856fb2256f037" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 33157 } }
I20251023 02:07:59.187525 16524 leader_election.cc:304] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0aefe06419b84268854856fb2256f037; no voters:
I20251023 02:07:59.188514 16521 ts_tablet_manager.cc:1434] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20251023 02:07:59.188709 16525 raft_consensus.cc:697] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 1 LEADER]: Becoming Leader. State: Replica: 0c761e8a5ed8401e8907a79efd8277f3, State: Running, Role: LEADER
I20251023 02:07:59.189023 16525 consensus_queue.cc:237] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c761e8a5ed8401e8907a79efd8277f3" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40971 } }
I20251023 02:07:59.189759 16520 leader_election.cc:290] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:07:59.189951 16529 raft_consensus.cc:2804] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:07:59.190624 16530 raft_consensus.cc:2804] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:07:59.190824 16524 leader_election.cc:290] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:07:59.191975 16530 raft_consensus.cc:697] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 1 LEADER]: Becoming Leader. State: Replica: 0aefe06419b84268854856fb2256f037, State: Running, Role: LEADER
I20251023 02:07:59.192384 16530 consensus_queue.cc:237] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aefe06419b84268854856fb2256f037" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 33157 } }
I20251023 02:07:59.193408 16524 ts_tablet_manager.cc:1434] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037: Time spent starting tablet: real 0.012s user 0.003s sys 0.003s
I20251023 02:07:59.195240 16520 ts_tablet_manager.cc:1434] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78: Time spent starting tablet: real 0.024s user 0.006s sys 0.016s
I20251023 02:07:59.195587 16520 tablet_bootstrap.cc:492] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78: Bootstrap starting.
I20251023 02:07:59.198582 16101 catalog_manager.cc:5649] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aefe06419b84268854856fb2256f037 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "0aefe06419b84268854856fb2256f037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aefe06419b84268854856fb2256f037" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 33157 } health_report { overall_health: HEALTHY } } }
I20251023 02:07:59.200834 16100 catalog_manager.cc:5649] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c761e8a5ed8401e8907a79efd8277f3 (127.15.161.194). New cstate: current_term: 1 leader_uuid: "0c761e8a5ed8401e8907a79efd8277f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c761e8a5ed8401e8907a79efd8277f3" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40971 } health_report { overall_health: HEALTHY } } }
I20251023 02:07:59.201967 16520 tablet_bootstrap.cc:654] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:59.208053 16529 raft_consensus.cc:697] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 LEADER]: Becoming Leader. State: Replica: 6f6e4dd23eb341bc82aa9d24a634ba78, State: Running, Role: LEADER
I20251023 02:07:59.208340 16529 consensus_queue.cc:237] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.214927 16101 catalog_manager.cc:5649] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f6e4dd23eb341bc82aa9d24a634ba78 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } health_report { overall_health: HEALTHY } } }
I20251023 02:07:59.246608 16520 tablet_bootstrap.cc:492] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78: No bootstrap required, opened a new log
I20251023 02:07:59.246803 16520 ts_tablet_manager.cc:1403] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.008s
I20251023 02:07:59.247432 16520 raft_consensus.cc:359] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.247596 16520 raft_consensus.cc:385] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:59.247670 16520 raft_consensus.cc:740] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6e4dd23eb341bc82aa9d24a634ba78, State: Initialized, Role: FOLLOWER
I20251023 02:07:59.247870 16520 consensus_queue.cc:260] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.248003 16520 raft_consensus.cc:399] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:07:59.248087 16520 raft_consensus.cc:493] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:07:59.248180 16520 raft_consensus.cc:3060] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:59.249811 16520 raft_consensus.cc:515] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.250003 16520 leader_election.cc:304] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f6e4dd23eb341bc82aa9d24a634ba78; no voters:
I20251023 02:07:59.250211 16520 leader_election.cc:290] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:07:59.250310 16533 raft_consensus.cc:2804] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:07:59.250474 16533 raft_consensus.cc:697] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 LEADER]: Becoming Leader. State: Replica: 6f6e4dd23eb341bc82aa9d24a634ba78, State: Running, Role: LEADER
I20251023 02:07:59.250643 16520 ts_tablet_manager.cc:1434] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251023 02:07:59.250746 16533 consensus_queue.cc:237] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } }
I20251023 02:07:59.252781 16101 catalog_manager.cc:5649] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f6e4dd23eb341bc82aa9d24a634ba78 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6e4dd23eb341bc82aa9d24a634ba78" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40877 } health_report { overall_health: HEALTHY } } }
I20251023 02:07:59.270740 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:07:59.281566 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:07:59.282075 16007 tablet_replica.cc:333] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78: stopping tablet replica
I20251023 02:07:59.282433 16007 raft_consensus.cc:2243] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:07:59.282675 16007 raft_consensus.cc:2272] T bf9536a59806474fa1e3e75cc97859e6 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.284060 16007 tablet_replica.cc:333] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78: stopping tablet replica
I20251023 02:07:59.284201 16007 raft_consensus.cc:2243] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:07:59.284329 16007 raft_consensus.cc:2272] T cd6d425395924da1b159363891228c78 P 6f6e4dd23eb341bc82aa9d24a634ba78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.289227 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:07:59.293920 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:07:59.302641 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:07:59.302974 16007 tablet_replica.cc:333] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3: stopping tablet replica
I20251023 02:07:59.303117 16007 raft_consensus.cc:2243] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:07:59.303247 16007 raft_consensus.cc:2272] T da79a38919894cfebbbb21b4f0373fb8 P 0c761e8a5ed8401e8907a79efd8277f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.307101 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:07:59.309319 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:07:59.317122 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:07:59.317415 16007 tablet_replica.cc:333] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037: stopping tablet replica
I20251023 02:07:59.317593 16007 raft_consensus.cc:2243] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:07:59.317746 16007 raft_consensus.cc:2272] T 2c55b3d4976a4d2c903405ab265bd87e P 0aefe06419b84268854856fb2256f037 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.331475 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:07:59.333755 16007 master.cc:561] Master@127.15.161.254:38547 shutting down...
I20251023 02:07:59.341017 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:07:59.341216 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.341359 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2af3410a78774d75b6665728ecb20530: stopping tablet replica
I20251023 02:07:59.355159 16007 master.cc:583] Master@127.15.161.254:38547 shutdown complete.
I20251023 02:07:59.358950 16007 master.cc:561] Master@127.15.161.253:39547 shutting down...
I20251023 02:07:59.365561 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:07:59.365988 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.366087 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 15bb4c0004234d44a4e232cc464ffad3: stopping tablet replica
I20251023 02:07:59.370663 16007 master.cc:583] Master@127.15.161.253:39547 shutdown complete.
I20251023 02:07:59.374606 16007 master.cc:561] Master@127.15.161.252:38849 shutting down...
I20251023 02:07:59.381460 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:07:59.381628 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:07:59.381719 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4be0e458870b43d394b91214fdbc4ab2: stopping tablet replica
I20251023 02:07:59.395164 16007 master.cc:583] Master@127.15.161.252:38849 shutdown complete.
[ OK ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing (3039 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn
I20251023 02:07:59.407972 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:44077
I20251023 02:07:59.408438 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:59.410529 16543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:59.410651 16544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:59.411073 16546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:59.411978 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:59.412210 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:59.412289 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:59.412341 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185279412324 us; error 0 us; skew 500 ppm
I20251023 02:07:59.412493 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:59.413509 16007 webserver.cc:492] Webserver started at http://127.15.161.254:39101/ using document root <none> and password file <none>
I20251023 02:07:59.413738 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:59.413813 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:59.413964 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:59.414566 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "5373aaa178d24db5bf9aaeb209b066f4"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.417025 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:07:59.418457 16551 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.418746 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:07:59.418848 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "5373aaa178d24db5bf9aaeb209b066f4"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.418956 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:59.434813 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:59.435346 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:59.448627 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:44077
I20251023 02:07:59.448645 16602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:44077 every 8 connection(s)
I20251023 02:07:59.449834 16603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:07:59.453248 16603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4: Bootstrap starting.
I20251023 02:07:59.454473 16603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4: Neither blocks nor log segments found. Creating new log.
I20251023 02:07:59.455868 16603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4: No bootstrap required, opened a new log
I20251023 02:07:59.456413 16603 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5373aaa178d24db5bf9aaeb209b066f4" member_type: VOTER }
I20251023 02:07:59.456548 16603 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:07:59.456621 16603 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5373aaa178d24db5bf9aaeb209b066f4, State: Initialized, Role: FOLLOWER
I20251023 02:07:59.456807 16603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5373aaa178d24db5bf9aaeb209b066f4" member_type: VOTER }
I20251023 02:07:59.456928 16603 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:07:59.456976 16603 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:07:59.457041 16603 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:07:59.458503 16603 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5373aaa178d24db5bf9aaeb209b066f4" member_type: VOTER }
I20251023 02:07:59.458678 16603 leader_election.cc:304] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5373aaa178d24db5bf9aaeb209b066f4; no voters:
I20251023 02:07:59.458992 16603 leader_election.cc:290] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:07:59.459115 16606 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:07:59.459618 16606 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 1 LEADER]: Becoming Leader. State: Replica: 5373aaa178d24db5bf9aaeb209b066f4, State: Running, Role: LEADER
I20251023 02:07:59.459805 16603 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:07:59.459874 16606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5373aaa178d24db5bf9aaeb209b066f4" member_type: VOTER }
I20251023 02:07:59.460723 16607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5373aaa178d24db5bf9aaeb209b066f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5373aaa178d24db5bf9aaeb209b066f4" member_type: VOTER } }
I20251023 02:07:59.460789 16608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5373aaa178d24db5bf9aaeb209b066f4. Latest consensus state: current_term: 1 leader_uuid: "5373aaa178d24db5bf9aaeb209b066f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5373aaa178d24db5bf9aaeb209b066f4" member_type: VOTER } }
I20251023 02:07:59.460981 16608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [sys.catalog]: This master's current role is: LEADER
I20251023 02:07:59.461006 16607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [sys.catalog]: This master's current role is: LEADER
I20251023 02:07:59.466413 16611 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:07:59.467967 16611 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:07:59.468394 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:07:59.470707 16611 catalog_manager.cc:1357] Generated new cluster ID: a017476d2d9e4072be67c683f33a5589
I20251023 02:07:59.470798 16611 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:07:59.483775 16611 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:07:59.484439 16611 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:07:59.491714 16611 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4: Generated new TSK 0
I20251023 02:07:59.491930 16611 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:07:59.500861 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:59.503369 16624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:59.503654 16625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:59.504544 16627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:59.504803 16007 server_base.cc:1047] running on GCE node
I20251023 02:07:59.505041 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:59.505111 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:59.505178 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185279505147 us; error 0 us; skew 500 ppm
I20251023 02:07:59.505344 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:59.506511 16007 webserver.cc:492] Webserver started at http://127.15.161.193:42813/ using document root <none> and password file <none>
I20251023 02:07:59.506727 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:59.506816 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:59.506940 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:59.507511 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "68f6ec407b1d402f8566ea125c0eb810"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.509706 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:07:59.511180 16632 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.511550 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:07:59.511637 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "68f6ec407b1d402f8566ea125c0eb810"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.511744 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:59.528249 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:59.528784 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:59.529258 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:07:59.530041 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:07:59.530107 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.530236 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:07:59.530293 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.544225 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:42593
I20251023 02:07:59.544258 16694 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:42593 every 8 connection(s)
I20251023 02:07:59.545774 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:59.548739 16699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:59.549028 16700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:59.549975 16007 server_base.cc:1047] running on GCE node
W20251023 02:07:59.550168 16702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:59.550614 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:59.550714 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:59.550779 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185279550757 us; error 0 us; skew 500 ppm
I20251023 02:07:59.550977 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:59.552273 16007 webserver.cc:492] Webserver started at http://127.15.161.194:44319/ using document root <none> and password file <none>
I20251023 02:07:59.552526 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:59.552624 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:59.552771 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:59.553369 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "246467b191014383b97905412e5ce7cf"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.553431 16695 heartbeater.cc:344] Connected to a master server at 127.15.161.254:44077
I20251023 02:07:59.553547 16695 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:59.553895 16695 heartbeater.cc:507] Master 127.15.161.254:44077 requested a full tablet report, sending...
I20251023 02:07:59.554694 16568 ts_manager.cc:194] Registered new tserver with Master: 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593)
I20251023 02:07:59.555735 16568 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40172
I20251023 02:07:59.555953 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:07:59.557404 16707 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.557734 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:07:59.557817 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "246467b191014383b97905412e5ce7cf"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.557897 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:59.570286 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:59.570750 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:59.571215 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:07:59.571900 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:07:59.571965 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.572023 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:07:59.572068 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.583781 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:40047
I20251023 02:07:59.583832 16769 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:40047 every 8 connection(s)
I20251023 02:07:59.585201 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:07:59.588264 16773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:07:59.588727 16774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:59.591507 16007 server_base.cc:1047] running on GCE node
W20251023 02:07:59.591879 16776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:07:59.592298 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:07:59.592391 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:07:59.592473 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185279592429 us; error 0 us; skew 500 ppm
I20251023 02:07:59.592628 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:07:59.592792 16770 heartbeater.cc:344] Connected to a master server at 127.15.161.254:44077
I20251023 02:07:59.592921 16770 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:59.593330 16770 heartbeater.cc:507] Master 127.15.161.254:44077 requested a full tablet report, sending...
I20251023 02:07:59.593752 16007 webserver.cc:492] Webserver started at http://127.15.161.195:33615/ using document root <none> and password file <none>
I20251023 02:07:59.593974 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:07:59.594074 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:07:59.594167 16568 ts_manager.cc:194] Registered new tserver with Master: 246467b191014383b97905412e5ce7cf (127.15.161.194:40047)
I20251023 02:07:59.594249 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:07:59.594853 16568 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40180
I20251023 02:07:59.594833 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "a8618ac81e1549178549865eb66bdfb4"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.597224 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20251023 02:07:59.598670 16781 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.598968 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 02:07:59.599074 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "a8618ac81e1549178549865eb66bdfb4"
format_stamp: "Formatted at 2025-10-23 02:07:59 on dist-test-slave-bfbt"
I20251023 02:07:59.599189 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:07:59.606001 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:07:59.606524 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:07:59.606982 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:07:59.607771 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:07:59.607841 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.607955 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:07:59.608009 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:07:59.620343 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:32909
I20251023 02:07:59.620395 16843 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:32909 every 8 connection(s)
I20251023 02:07:59.626257 16844 heartbeater.cc:344] Connected to a master server at 127.15.161.254:44077
I20251023 02:07:59.626390 16844 heartbeater.cc:461] Registering TS with master...
I20251023 02:07:59.626777 16844 heartbeater.cc:507] Master 127.15.161.254:44077 requested a full tablet report, sending...
I20251023 02:07:59.627506 16568 ts_manager.cc:194] Registered new tserver with Master: a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:07:59.627951 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006816425s
I20251023 02:07:59.628228 16568 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40184
I20251023 02:08:00.556950 16695 heartbeater.cc:499] Master 127.15.161.254:44077 was elected leader, sending a full tablet report...
I20251023 02:08:00.595992 16770 heartbeater.cc:499] Master 127.15.161.254:44077 was elected leader, sending a full tablet report...
I20251023 02:08:00.629390 16844 heartbeater.cc:499] Master 127.15.161.254:44077 was elected leader, sending a full tablet report...
I20251023 02:08:00.655393 16007 test_util.cc:276] Using random seed: -418807808
I20251023 02:08:00.668397 16568 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40190:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o""\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:08:00.669162 16568 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:08:00.703274 16735 tablet_service.cc:1505] Processing CreateTablet for tablet 7af85c4409a641d593757e92cb737877 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251023 02:08:00.703874 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7af85c4409a641d593757e92cb737877. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.703766 16659 tablet_service.cc:1505] Processing CreateTablet for tablet eae456735a20434a862c20ccea930c4b (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251023 02:08:00.704324 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eae456735a20434a862c20ccea930c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.704372 16807 tablet_service.cc:1505] Processing CreateTablet for tablet c3a8c5297ca846d7acd98685392ac164 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251023 02:08:00.704614 16660 tablet_service.cc:1505] Processing CreateTablet for tablet 7af85c4409a641d593757e92cb737877 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251023 02:08:00.704841 16807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3a8c5297ca846d7acd98685392ac164. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.705140 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7af85c4409a641d593757e92cb737877. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.706372 16733 tablet_service.cc:1505] Processing CreateTablet for tablet c3a8c5297ca846d7acd98685392ac164 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251023 02:08:00.706885 16733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3a8c5297ca846d7acd98685392ac164. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.710569 16731 tablet_service.cc:1505] Processing CreateTablet for tablet c036f3aca6224ae3b725c72e2aafc162 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251023 02:08:00.710638 16732 tablet_service.cc:1505] Processing CreateTablet for tablet 5d59c893ae0747279b76ee40c85f2ec4 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251023 02:08:00.703794 16808 tablet_service.cc:1505] Processing CreateTablet for tablet eae456735a20434a862c20ccea930c4b (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251023 02:08:00.711334 16808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eae456735a20434a862c20ccea930c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.711264 16656 tablet_service.cc:1505] Processing CreateTablet for tablet c036f3aca6224ae3b725c72e2aafc162 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251023 02:08:00.711515 16806 tablet_service.cc:1505] Processing CreateTablet for tablet 5d59c893ae0747279b76ee40c85f2ec4 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251023 02:08:00.711709 16656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c036f3aca6224ae3b725c72e2aafc162. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.711935 16806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d59c893ae0747279b76ee40c85f2ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.711877 16802 tablet_service.cc:1505] Processing CreateTablet for tablet d481d3550dff46dd95f16a66d80ddfe5 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251023 02:08:00.712302 16802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481d3550dff46dd95f16a66d80ddfe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.714108 16655 tablet_service.cc:1505] Processing CreateTablet for tablet 2a631d579ece4a1fa4a7210a36a992ab (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251023 02:08:00.714627 16655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a631d579ece4a1fa4a7210a36a992ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.714529 16803 tablet_service.cc:1505] Processing CreateTablet for tablet d0c1212f78ee4862a7df030fab315c99 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251023 02:08:00.714999 16803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c1212f78ee4862a7df030fab315c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.714902 16729 tablet_service.cc:1505] Processing CreateTablet for tablet d0c1212f78ee4862a7df030fab315c99 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251023 02:08:00.717250 16804 tablet_service.cc:1505] Processing CreateTablet for tablet 2a631d579ece4a1fa4a7210a36a992ab (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251023 02:08:00.717695 16804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a631d579ece4a1fa4a7210a36a992ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.718605 16654 tablet_service.cc:1505] Processing CreateTablet for tablet d0c1212f78ee4862a7df030fab315c99 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251023 02:08:00.719107 16654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c1212f78ee4862a7df030fab315c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.722648 16734 tablet_service.cc:1505] Processing CreateTablet for tablet eae456735a20434a862c20ccea930c4b (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251023 02:08:00.723026 16728 tablet_service.cc:1505] Processing CreateTablet for tablet d481d3550dff46dd95f16a66d80ddfe5 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251023 02:08:00.723596 16653 tablet_service.cc:1505] Processing CreateTablet for tablet d481d3550dff46dd95f16a66d80ddfe5 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251023 02:08:00.723742 16805 tablet_service.cc:1505] Processing CreateTablet for tablet c036f3aca6224ae3b725c72e2aafc162 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251023 02:08:00.724207 16805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c036f3aca6224ae3b725c72e2aafc162. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.724565 16658 tablet_service.cc:1505] Processing CreateTablet for tablet c3a8c5297ca846d7acd98685392ac164 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251023 02:08:00.726300 16809 tablet_service.cc:1505] Processing CreateTablet for tablet 7af85c4409a641d593757e92cb737877 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251023 02:08:00.726910 16809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7af85c4409a641d593757e92cb737877. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.724205 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481d3550dff46dd95f16a66d80ddfe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.730682 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3a8c5297ca846d7acd98685392ac164. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.740568 16864 tablet_bootstrap.cc:492] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.748608 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c036f3aca6224ae3b725c72e2aafc162. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.750240 16865 tablet_bootstrap.cc:492] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.751219 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d59c893ae0747279b76ee40c85f2ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.751487 16729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c1212f78ee4862a7df030fab315c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.751691 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eae456735a20434a862c20ccea930c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.751220 16730 tablet_service.cc:1505] Processing CreateTablet for tablet 2a631d579ece4a1fa4a7210a36a992ab (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251023 02:08:00.753069 16728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481d3550dff46dd95f16a66d80ddfe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.750965 16657 tablet_service.cc:1505] Processing CreateTablet for tablet 5d59c893ae0747279b76ee40c85f2ec4 (DEFAULT_TABLE table=test-workload [id=737b41cc6313449bb61b6c53a864874e]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251023 02:08:00.756526 16730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a631d579ece4a1fa4a7210a36a992ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.756713 16657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d59c893ae0747279b76ee40c85f2ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:00.768498 16864 tablet_bootstrap.cc:654] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.770145 16866 tablet_bootstrap.cc:492] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.776652 16865 tablet_bootstrap.cc:654] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.781212 16865 tablet_bootstrap.cc:492] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.781422 16865 ts_tablet_manager.cc:1403] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.000s
I20251023 02:08:00.781522 16866 tablet_bootstrap.cc:654] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.781970 16864 tablet_bootstrap.cc:492] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.782176 16864 ts_tablet_manager.cc:1403] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.007s
I20251023 02:08:00.782181 16865 raft_consensus.cc:359] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.782442 16865 raft_consensus.cc:385] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.782558 16865 raft_consensus.cc:740] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.782792 16865 consensus_queue.cc:260] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.783296 16864 raft_consensus.cc:359] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.783842 16864 raft_consensus.cc:385] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.783847 16865 ts_tablet_manager.cc:1434] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:08:00.784040 16864 raft_consensus.cc:740] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.784154 16865 tablet_bootstrap.cc:492] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.784330 16864 consensus_queue.cc:260] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.786281 16864 ts_tablet_manager.cc:1434] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20251023 02:08:00.786607 16864 tablet_bootstrap.cc:492] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.789156 16864 tablet_bootstrap.cc:654] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.791989 16865 tablet_bootstrap.cc:654] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.793931 16864 tablet_bootstrap.cc:492] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.794113 16864 ts_tablet_manager.cc:1403] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20251023 02:08:00.794199 16866 tablet_bootstrap.cc:492] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.794345 16866 ts_tablet_manager.cc:1403] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.009s
I20251023 02:08:00.794548 16865 tablet_bootstrap.cc:492] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.794720 16865 ts_tablet_manager.cc:1403] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.003s
I20251023 02:08:00.795113 16866 raft_consensus.cc:359] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.795120 16864 raft_consensus.cc:359] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.795317 16866 raft_consensus.cc:385] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.795367 16864 raft_consensus.cc:385] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.795464 16866 raft_consensus.cc:740] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.795490 16864 raft_consensus.cc:740] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.795552 16865 raft_consensus.cc:359] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:00.795756 16865 raft_consensus.cc:385] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.795723 16866 consensus_queue.cc:260] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.795867 16865 raft_consensus.cc:740] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.795841 16864 consensus_queue.cc:260] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.796128 16865 consensus_queue.cc:260] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:00.796304 16864 ts_tablet_manager.cc:1434] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:00.796906 16866 ts_tablet_manager.cc:1434] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:08:00.797159 16866 tablet_bootstrap.cc:492] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.797354 16864 tablet_bootstrap.cc:492] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.797399 16865 ts_tablet_manager.cc:1434] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:08:00.797737 16865 tablet_bootstrap.cc:492] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.799074 16866 tablet_bootstrap.cc:654] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.799407 16864 tablet_bootstrap.cc:654] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.799624 16865 tablet_bootstrap.cc:654] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.804896 16865 tablet_bootstrap.cc:492] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.805088 16865 ts_tablet_manager.cc:1403] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251023 02:08:00.806037 16865 raft_consensus.cc:359] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.806303 16865 raft_consensus.cc:385] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.806519 16865 raft_consensus.cc:740] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.806821 16865 consensus_queue.cc:260] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.807376 16865 ts_tablet_manager.cc:1434] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 02:08:00.807657 16864 tablet_bootstrap.cc:492] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.807879 16864 ts_tablet_manager.cc:1403] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20251023 02:08:00.807658 16865 tablet_bootstrap.cc:492] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.808591 16864 raft_consensus.cc:359] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.808728 16866 tablet_bootstrap.cc:492] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.808776 16864 raft_consensus.cc:385] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.808925 16866 ts_tablet_manager.cc:1403] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20251023 02:08:00.809022 16864 raft_consensus.cc:740] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.809386 16864 consensus_queue.cc:260] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.809641 16866 raft_consensus.cc:359] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.809880 16866 raft_consensus.cc:385] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.809903 16865 tablet_bootstrap.cc:654] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.809978 16866 raft_consensus.cc:740] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.810268 16866 consensus_queue.cc:260] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.811301 16866 ts_tablet_manager.cc:1434] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:08:00.811604 16866 tablet_bootstrap.cc:492] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.812446 16864 ts_tablet_manager.cc:1434] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251023 02:08:00.813184 16866 tablet_bootstrap.cc:654] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.813534 16864 tablet_bootstrap.cc:492] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.816576 16864 tablet_bootstrap.cc:654] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.818845 16865 tablet_bootstrap.cc:492] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.818957 16870 raft_consensus.cc:493] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:00.819020 16865 ts_tablet_manager.cc:1403] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20251023 02:08:00.819095 16870 raft_consensus.cc:515] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.819759 16865 raft_consensus.cc:359] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.819931 16865 raft_consensus.cc:385] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.820021 16865 raft_consensus.cc:740] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.820155 16870 leader_election.cc:290] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.820214 16865 consensus_queue.cc:260] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.822043 16866 tablet_bootstrap.cc:492] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.822327 16871 raft_consensus.cc:493] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:00.822499 16871 raft_consensus.cc:515] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.818845 16864 tablet_bootstrap.cc:492] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.823271 16864 ts_tablet_manager.cc:1403] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20251023 02:08:00.823591 16871 leader_election.cc:290] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 246467b191014383b97905412e5ce7cf (127.15.161.194:40047), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.824944 16864 raft_consensus.cc:359] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.825095 16866 ts_tablet_manager.cc:1403] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20251023 02:08:00.825157 16864 raft_consensus.cc:385] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.825266 16864 raft_consensus.cc:740] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.825505 16864 consensus_queue.cc:260] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.825986 16866 raft_consensus.cc:359] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.826244 16866 raft_consensus.cc:385] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.826386 16866 raft_consensus.cc:740] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.826700 16866 consensus_queue.cc:260] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.827183 16866 ts_tablet_manager.cc:1434] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:08:00.827541 16865 ts_tablet_manager.cc:1434] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251023 02:08:00.828841 16864 ts_tablet_manager.cc:1434] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251023 02:08:00.829198 16864 tablet_bootstrap.cc:492] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.829727 16865 tablet_bootstrap.cc:492] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.831280 16866 tablet_bootstrap.cc:492] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.831772 16865 tablet_bootstrap.cc:654] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.832559 16864 tablet_bootstrap.cc:654] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.833063 16866 tablet_bootstrap.cc:654] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.836225 16864 tablet_bootstrap.cc:492] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.836410 16864 ts_tablet_manager.cc:1403] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251023 02:08:00.836867 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c036f3aca6224ae3b725c72e2aafc162" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf" is_pre_election: true
I20251023 02:08:00.837359 16864 raft_consensus.cc:359] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.837572 16864 raft_consensus.cc:385] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.837680 16864 raft_consensus.cc:740] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
W20251023 02:08:00.837960 16635 leader_election.cc:343] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 246467b191014383b97905412e5ce7cf (127.15.161.194:40047): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.841033 16866 tablet_bootstrap.cc:492] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.841208 16866 ts_tablet_manager.cc:1403] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.002s
I20251023 02:08:00.842049 16866 raft_consensus.cc:359] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:00.842303 16866 raft_consensus.cc:385] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.842402 16866 raft_consensus.cc:740] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.842716 16866 consensus_queue.cc:260] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:00.843254 16865 tablet_bootstrap.cc:492] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.843392 16865 ts_tablet_manager.cc:1403] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20251023 02:08:00.844074 16865 raft_consensus.cc:359] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.844241 16865 raft_consensus.cc:385] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.844343 16865 raft_consensus.cc:740] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.844538 16865 consensus_queue.cc:260] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.845012 16865 ts_tablet_manager.cc:1434] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 02:08:00.845976 16866 ts_tablet_manager.cc:1434] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20251023 02:08:00.847070 16866 tablet_bootstrap.cc:492] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.849258 16866 tablet_bootstrap.cc:654] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.850543 16865 tablet_bootstrap.cc:492] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.852290 16865 tablet_bootstrap.cc:654] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.855333 16819 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:00.855628 16865 tablet_bootstrap.cc:492] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.855690 16819 raft_consensus.cc:2468] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:00.855798 16865 ts_tablet_manager.cc:1403] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251023 02:08:00.856221 16708 leader_election.cc:304] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:00.856501 16870 raft_consensus.cc:2804] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:00.856631 16870 raft_consensus.cc:493] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:00.856746 16870 raft_consensus.cc:3060] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.856721 16865 raft_consensus.cc:359] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.856909 16865 raft_consensus.cc:385] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.856993 16865 raft_consensus.cc:740] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.857218 16865 consensus_queue.cc:260] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.860008 16870 raft_consensus.cc:515] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.860953 16870 leader_election.cc:290] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.861676 16819 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:00.861866 16819 raft_consensus.cc:3060] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.837942 16864 consensus_queue.cc:260] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.865140 16865 ts_tablet_manager.cc:1434] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.009s user 0.005s sys 0.005s
I20251023 02:08:00.865283 16864 ts_tablet_manager.cc:1434] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.029s user 0.002s sys 0.001s
I20251023 02:08:00.865742 16864 tablet_bootstrap.cc:492] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.866180 16669 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
W20251023 02:08:00.866942 16711 leader_election.cc:343] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.867161 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:00.867722 16819 raft_consensus.cc:2468] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:00.868098 16708 leader_election.cc:304] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 246467b191014383b97905412e5ce7cf, a8618ac81e1549178549865eb66bdfb4; no voters: 68f6ec407b1d402f8566ea125c0eb810
W20251023 02:08:00.868870 16711 leader_election.cc:343] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.868920 16818 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c036f3aca6224ae3b725c72e2aafc162" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:00.869026 16870 raft_consensus.cc:2804] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:00.869302 16870 raft_consensus.cc:697] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Leader. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:00.869503 16864 tablet_bootstrap.cc:654] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
W20251023 02:08:00.869625 16633 leader_election.cc:343] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.869598 16870 consensus_queue.cc:237] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.869755 16633 leader_election.cc:304] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68f6ec407b1d402f8566ea125c0eb810; no voters: 246467b191014383b97905412e5ce7cf, a8618ac81e1549178549865eb66bdfb4
I20251023 02:08:00.870046 16871 raft_consensus.cc:2749] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:08:00.872814 16870 raft_consensus.cc:493] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:00.872957 16870 raft_consensus.cc:515] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.873522 16870 leader_election.cc:290] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.874241 16566 catalog_manager.cc:5649] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 246467b191014383b97905412e5ce7cf (127.15.161.194). New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:00.878682 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:00.879130 16670 raft_consensus.cc:2468] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:00.879396 16819 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
W20251023 02:08:00.879854 16708 leader_election.cc:343] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.880010 16711 leader_election.cc:304] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters: a8618ac81e1549178549865eb66bdfb4
I20251023 02:08:00.880293 16864 tablet_bootstrap.cc:492] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.880447 16864 ts_tablet_manager.cc:1403] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20251023 02:08:00.881114 16866 tablet_bootstrap.cc:492] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.881170 16894 raft_consensus.cc:2804] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:00.881281 16894 raft_consensus.cc:493] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:00.881311 16866 ts_tablet_manager.cc:1403] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.019s
I20251023 02:08:00.881259 16864 raft_consensus.cc:359] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:00.881400 16894 raft_consensus.cc:3060] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.881460 16864 raft_consensus.cc:385] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.881559 16864 raft_consensus.cc:740] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.881871 16864 consensus_queue.cc:260] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:00.882294 16866 raft_consensus.cc:359] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.882479 16864 ts_tablet_manager.cc:1434] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:08:00.882510 16866 raft_consensus.cc:385] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.882638 16865 tablet_bootstrap.cc:492] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.882668 16866 raft_consensus.cc:740] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.882761 16864 tablet_bootstrap.cc:492] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.883033 16866 consensus_queue.cc:260] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.883692 16866 ts_tablet_manager.cc:1434] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:08:00.884016 16866 tablet_bootstrap.cc:492] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.884403 16865 tablet_bootstrap.cc:654] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.884475 16864 tablet_bootstrap.cc:654] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.886010 16866 tablet_bootstrap.cc:654] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.887396 16891 raft_consensus.cc:493] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:00.887635 16891 raft_consensus.cc:515] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.888892 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0c1212f78ee4862a7df030fab315c99" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:00.889101 16670 raft_consensus.cc:2468] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:00.889197 16819 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0c1212f78ee4862a7df030fab315c99" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:00.889506 16711 leader_election.cc:304] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
W20251023 02:08:00.889667 16708 leader_election.cc:343] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.889791 16870 raft_consensus.cc:2804] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:00.889839 16891 leader_election.cc:290] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.889911 16870 raft_consensus.cc:493] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:00.890005 16870 raft_consensus.cc:3060] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.892856 16864 tablet_bootstrap.cc:492] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.893023 16864 ts_tablet_manager.cc:1403] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20251023 02:08:00.893503 16870 raft_consensus.cc:515] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.893986 16864 raft_consensus.cc:359] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.894246 16864 raft_consensus.cc:385] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.894356 16864 raft_consensus.cc:740] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.894683 16864 consensus_queue.cc:260] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.894776 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0c1212f78ee4862a7df030fab315c99" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:00.895007 16670 raft_consensus.cc:3060] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.896302 16864 ts_tablet_manager.cc:1434] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20251023 02:08:00.896685 16864 tablet_bootstrap.cc:492] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810: Bootstrap starting.
I20251023 02:08:00.898869 16670 raft_consensus.cc:2468] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:00.898952 16864 tablet_bootstrap.cc:654] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.899379 16711 leader_election.cc:304] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:00.899721 16891 raft_consensus.cc:2804] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:00.901185 16866 tablet_bootstrap.cc:492] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.901376 16866 ts_tablet_manager.cc:1403] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.007s
I20251023 02:08:00.901455 16864 tablet_bootstrap.cc:492] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810: No bootstrap required, opened a new log
I20251023 02:08:00.901605 16864 ts_tablet_manager.cc:1403] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 02:08:00.902268 16866 raft_consensus.cc:359] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.902284 16864 raft_consensus.cc:359] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.902530 16866 raft_consensus.cc:385] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.902550 16864 raft_consensus.cc:385] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.902694 16866 raft_consensus.cc:740] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.902698 16864 raft_consensus.cc:740] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.903029 16864 consensus_queue.cc:260] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.903050 16866 consensus_queue.cc:260] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.903738 16864 ts_tablet_manager.cc:1434] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:00.904289 16866 ts_tablet_manager.cc:1434] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:08:00.904659 16866 tablet_bootstrap.cc:492] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.906306 16866 tablet_bootstrap.cc:654] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.908214 16870 leader_election.cc:290] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.908456 16870 raft_consensus.cc:493] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:00.908659 16870 raft_consensus.cc:515] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.909307 16870 leader_election.cc:290] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.909713 16669 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c036f3aca6224ae3b725c72e2aafc162" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:00.909952 16669 raft_consensus.cc:2468] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:00.910485 16711 leader_election.cc:304] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:00.912406 16870 raft_consensus.cc:2804] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:00.912511 16870 raft_consensus.cc:493] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:00.912642 16870 raft_consensus.cc:3060] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.914297 16866 tablet_bootstrap.cc:492] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.914459 16866 ts_tablet_manager.cc:1403] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20251023 02:08:00.915055 16866 raft_consensus.cc:359] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.915717 16866 raft_consensus.cc:385] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.915824 16866 raft_consensus.cc:740] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.916079 16866 consensus_queue.cc:260] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.916572 16866 ts_tablet_manager.cc:1434] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 02:08:00.916872 16866 tablet_bootstrap.cc:492] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4: Bootstrap starting.
I20251023 02:08:00.917577 16870 raft_consensus.cc:515] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.918648 16866 tablet_bootstrap.cc:654] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.923147 16894 raft_consensus.cc:515] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.924274 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:00.924490 16670 raft_consensus.cc:3060] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.927943 16670 raft_consensus.cc:2468] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:00.928426 16711 leader_election.cc:304] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:00.931198 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c036f3aca6224ae3b725c72e2aafc162" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:00.931407 16670 raft_consensus.cc:3060] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.934490 16870 leader_election.cc:290] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.934708 16870 raft_consensus.cc:2804] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:00.934859 16870 raft_consensus.cc:697] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Leader. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:00.935164 16870 consensus_queue.cc:237] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.936699 16891 raft_consensus.cc:697] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Leader. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:00.936961 16891 consensus_queue.cc:237] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.938077 16894 leader_election.cc:290] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.939721 16670 raft_consensus.cc:2468] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:00.940227 16711 leader_election.cc:304] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:00.940536 16894 raft_consensus.cc:2804] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:00.940735 16894 raft_consensus.cc:697] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Leader. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:00.941020 16894 consensus_queue.cc:237] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.944192 16566 catalog_manager.cc:5649] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 246467b191014383b97905412e5ce7cf (127.15.161.194). New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:00.952464 16819 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0c1212f78ee4862a7df030fab315c99" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:00.952893 16818 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c036f3aca6224ae3b725c72e2aafc162" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:00.953634 16818 raft_consensus.cc:2468] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
W20251023 02:08:00.954195 16708 leader_election.cc:343] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:08:00.954860 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c036f3aca6224ae3b725c72e2aafc162" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:00.955315 16817 raft_consensus.cc:3060] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.955685 16818 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:00.956063 16818 raft_consensus.cc:3060] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.961943 16817 raft_consensus.cc:2468] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:00.963099 16818 raft_consensus.cc:2468] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:00.965476 16865 tablet_bootstrap.cc:492] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.965618 16865 ts_tablet_manager.cc:1403] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.091s user 0.015s sys 0.044s
I20251023 02:08:00.965955 16866 tablet_bootstrap.cc:492] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4: No bootstrap required, opened a new log
I20251023 02:08:00.965854 16566 catalog_manager.cc:5649] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 246467b191014383b97905412e5ce7cf (127.15.161.194). New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:00.966145 16866 ts_tablet_manager.cc:1403] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4: Time spent bootstrapping tablet: real 0.049s user 0.014s sys 0.033s
I20251023 02:08:00.966375 16865 raft_consensus.cc:359] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.966512 16865 raft_consensus.cc:385] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.966660 16865 raft_consensus.cc:740] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.966491 16566 catalog_manager.cc:5649] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 246467b191014383b97905412e5ce7cf (127.15.161.194). New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:00.966867 16865 consensus_queue.cc:260] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.966917 16866 raft_consensus.cc:359] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.967154 16866 raft_consensus.cc:385] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.967242 16866 raft_consensus.cc:740] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.967303 16865 ts_tablet_manager.cc:1434] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:00.967581 16865 tablet_bootstrap.cc:492] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: Bootstrap starting.
I20251023 02:08:00.967478 16866 consensus_queue.cc:260] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.967952 16866 ts_tablet_manager.cc:1434] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 02:08:00.969916 16865 tablet_bootstrap.cc:654] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:00.971846 16865 tablet_bootstrap.cc:492] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: No bootstrap required, opened a new log
I20251023 02:08:00.971989 16865 ts_tablet_manager.cc:1403] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:08:00.972482 16865 raft_consensus.cc:359] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.972623 16865 raft_consensus.cc:385] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:00.972692 16865 raft_consensus.cc:740] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Initialized, Role: FOLLOWER
I20251023 02:08:00.972875 16865 consensus_queue.cc:260] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.973212 16865 ts_tablet_manager.cc:1434] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:08:00.986366 16871 raft_consensus.cc:493] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:00.986534 16871 raft_consensus.cc:515] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.987074 16871 leader_election.cc:290] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 246467b191014383b97905412e5ce7cf (127.15.161.194:40047), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.987427 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eae456735a20434a862c20ccea930c4b" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf" is_pre_election: true
I20251023 02:08:00.987471 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eae456735a20434a862c20ccea930c4b" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:00.987603 16745 raft_consensus.cc:2468] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 0.
I20251023 02:08:00.987655 16817 raft_consensus.cc:2468] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 0.
I20251023 02:08:00.987937 16635 leader_election.cc:304] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:00.988145 16871 raft_consensus.cc:2804] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:00.988245 16871 raft_consensus.cc:493] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:00.988327 16871 raft_consensus.cc:3060] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.990067 16871 raft_consensus.cc:515] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.990537 16871 leader_election.cc:290] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 election: Requested vote from peers 246467b191014383b97905412e5ce7cf (127.15.161.194:40047), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:00.990878 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eae456735a20434a862c20ccea930c4b" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf"
I20251023 02:08:00.990880 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eae456735a20434a862c20ccea930c4b" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:00.991098 16745 raft_consensus.cc:3060] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.991120 16817 raft_consensus.cc:3060] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:00.993656 16817 raft_consensus.cc:2468] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 1.
I20251023 02:08:00.993654 16745 raft_consensus.cc:2468] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 1.
I20251023 02:08:00.994035 16633 leader_election.cc:304] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:00.994297 16871 raft_consensus.cc:2804] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:00.994680 16871 raft_consensus.cc:697] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 1 LEADER]: Becoming Leader. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Running, Role: LEADER
I20251023 02:08:00.994921 16871 consensus_queue.cc:237] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:00.997810 16567 catalog_manager.cc:5649] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "68f6ec407b1d402f8566ea125c0eb810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:01.044736 16894 raft_consensus.cc:493] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:01.044950 16894 raft_consensus.cc:515] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:01.045717 16894 leader_election.cc:290] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909), 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593)
I20251023 02:08:01.046252 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7af85c4409a641d593757e92cb737877" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:01.046277 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7af85c4409a641d593757e92cb737877" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:01.046450 16817 raft_consensus.cc:2468] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:01.046489 16670 raft_consensus.cc:2468] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:01.046891 16708 leader_election.cc:304] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:01.047174 16894 raft_consensus.cc:2804] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:01.047277 16894 raft_consensus.cc:493] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:01.047348 16894 raft_consensus.cc:3060] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.049257 16894 raft_consensus.cc:515] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:01.049772 16894 leader_election.cc:290] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Requested vote from peers a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909), 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593)
I20251023 02:08:01.050226 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7af85c4409a641d593757e92cb737877" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:01.050272 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7af85c4409a641d593757e92cb737877" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:01.050396 16817 raft_consensus.cc:3060] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.050418 16670 raft_consensus.cc:3060] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.052386 16670 raft_consensus.cc:2468] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:01.052410 16817 raft_consensus.cc:2468] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:01.052784 16711 leader_election.cc:304] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:01.053030 16894 raft_consensus.cc:2804] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:01.053179 16894 raft_consensus.cc:697] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Leader. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:01.053452 16894 consensus_queue.cc:237] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } }
I20251023 02:08:01.056092 16567 catalog_manager.cc:5649] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 246467b191014383b97905412e5ce7cf (127.15.161.194). New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:01.077299 16894 raft_consensus.cc:493] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:01.077481 16894 raft_consensus.cc:515] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.078071 16894 leader_election.cc:290] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:01.078558 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:01.078562 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:01.078807 16817 raft_consensus.cc:2468] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:01.078830 16670 raft_consensus.cc:2468] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 0.
I20251023 02:08:01.079188 16711 leader_election.cc:304] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:01.079413 16894 raft_consensus.cc:2804] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:01.079501 16894 raft_consensus.cc:493] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:01.079604 16894 raft_consensus.cc:3060] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.081808 16894 raft_consensus.cc:515] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.082376 16894 leader_election.cc:290] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:01.082736 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:01.082789 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4" candidate_uuid: "246467b191014383b97905412e5ce7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:01.082964 16817 raft_consensus.cc:3060] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.082985 16670 raft_consensus.cc:3060] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.085714 16817 raft_consensus.cc:2468] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:01.085714 16670 raft_consensus.cc:2468] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246467b191014383b97905412e5ce7cf in term 1.
I20251023 02:08:01.086153 16711 leader_election.cc:304] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810; no voters:
I20251023 02:08:01.086560 16894 raft_consensus.cc:2804] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:01.086755 16894 raft_consensus.cc:697] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Leader. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:01.086989 16894 consensus_queue.cc:237] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.089291 16567 catalog_manager.cc:5649] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 246467b191014383b97905412e5ce7cf (127.15.161.194). New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:01.165618 16874 raft_consensus.cc:493] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:01.165813 16874 raft_consensus.cc:515] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.166674 16874 leader_election.cc:290] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), 246467b191014383b97905412e5ce7cf (127.15.161.194:40047)
I20251023 02:08:01.173722 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d481d3550dff46dd95f16a66d80ddfe5" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:01.173857 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d481d3550dff46dd95f16a66d80ddfe5" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf" is_pre_election: true
I20251023 02:08:01.173964 16670 raft_consensus.cc:2468] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8618ac81e1549178549865eb66bdfb4 in term 0.
I20251023 02:08:01.174052 16745 raft_consensus.cc:2468] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8618ac81e1549178549865eb66bdfb4 in term 0.
I20251023 02:08:01.174433 16785 leader_election.cc:304] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:01.174698 16874 raft_consensus.cc:2804] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:01.174795 16874 raft_consensus.cc:493] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:01.174886 16874 raft_consensus.cc:3060] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.176746 16874 raft_consensus.cc:515] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.177228 16874 leader_election.cc:290] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 1 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), 246467b191014383b97905412e5ce7cf (127.15.161.194:40047)
I20251023 02:08:01.177552 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d481d3550dff46dd95f16a66d80ddfe5" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:01.177616 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d481d3550dff46dd95f16a66d80ddfe5" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf"
I20251023 02:08:01.177717 16670 raft_consensus.cc:3060] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.177750 16745 raft_consensus.cc:3060] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.179530 16670 raft_consensus.cc:2468] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8618ac81e1549178549865eb66bdfb4 in term 1.
I20251023 02:08:01.179551 16745 raft_consensus.cc:2468] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8618ac81e1549178549865eb66bdfb4 in term 1.
I20251023 02:08:01.179883 16784 leader_election.cc:304] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:01.180117 16874 raft_consensus.cc:2804] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:01.180447 16874 raft_consensus.cc:697] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 1 LEADER]: Becoming Leader. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Running, Role: LEADER
I20251023 02:08:01.180732 16874 consensus_queue.cc:237] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.182915 16567 catalog_manager.cc:5649] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8618ac81e1549178549865eb66bdfb4 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "a8618ac81e1549178549865eb66bdfb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: HEALTHY } } }
I20251023 02:08:01.257400 16670 raft_consensus.cc:3060] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.259527 16871 raft_consensus.cc:493] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:01.259717 16871 raft_consensus.cc:515] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.259995 16894 consensus_queue.cc:1048] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.260365 16871 leader_election.cc:290] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 246467b191014383b97905412e5ce7cf (127.15.161.194:40047), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:01.260723 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf" is_pre_election: true
I20251023 02:08:01.260811 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4" is_pre_election: true
I20251023 02:08:01.260998 16817 raft_consensus.cc:2468] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 1.
I20251023 02:08:01.261440 16633 leader_election.cc:304] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters: 246467b191014383b97905412e5ce7cf
I20251023 02:08:01.262226 16871 raft_consensus.cc:2804] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:08:01.262332 16871 raft_consensus.cc:493] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 246467b191014383b97905412e5ce7cf)
I20251023 02:08:01.262485 16871 raft_consensus.cc:3060] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.265094 16871 raft_consensus.cc:515] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.265406 16670 raft_consensus.cc:1240] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Rejecting Update request from peer 246467b191014383b97905412e5ce7cf for earlier term 1. Current term is 2. Ops: []
I20251023 02:08:01.265954 16900 consensus_queue.cc:1059] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.266006 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf"
I20251023 02:08:01.266242 16870 raft_consensus.cc:3055] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:08:01.266580 16870 raft_consensus.cc:740] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:01.266851 16871 leader_election.cc:290] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 2 election: Requested vote from peers 246467b191014383b97905412e5ce7cf (127.15.161.194:40047), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:01.266811 16900 consensus_queue.cc:1048] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.267144 16870 consensus_queue.cc:260] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.267403 16817 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3a8c5297ca846d7acd98685392ac164" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:01.267583 16817 raft_consensus.cc:3060] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.267580 16870 raft_consensus.cc:3060] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.270361 16817 raft_consensus.cc:2468] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 2.
I20251023 02:08:01.270704 16818 raft_consensus.cc:1240] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Rejecting Update request from peer 246467b191014383b97905412e5ce7cf for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20251023 02:08:01.270747 16633 leader_election.cc:304] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters: 246467b191014383b97905412e5ce7cf
I20251023 02:08:01.270993 16871 raft_consensus.cc:2804] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:08:01.271154 16871 raft_consensus.cc:697] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 LEADER]: Becoming Leader. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Running, Role: LEADER
I20251023 02:08:01.271404 16871 consensus_queue.cc:237] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.272972 16567 catalog_manager.cc:5649] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 reported cstate change: term changed from 1 to 2, leader changed from 246467b191014383b97905412e5ce7cf (127.15.161.194) to 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193). New cstate: current_term: 2 leader_uuid: "68f6ec407b1d402f8566ea125c0eb810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:01.284734 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:01.287322 16917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:01.287369 16918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:01.288539 16920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:01.288741 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:01.289438 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:01.289512 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:01.289564 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185281289548 us; error 0 us; skew 500 ppm
I20251023 02:08:01.289716 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:01.290863 16007 webserver.cc:492] Webserver started at http://127.15.161.196:33681/ using document root <none> and password file <none>
I20251023 02:08:01.291092 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:01.291170 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:01.291324 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:01.291942 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "0ba79ef4fac849aa911b2f4679b8120d"
format_stamp: "Formatted at 2025-10-23 02:08:01 on dist-test-slave-bfbt"
I20251023 02:08:01.294505 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:08:01.296010 16925 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:01.296341 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:01.296450 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "0ba79ef4fac849aa911b2f4679b8120d"
format_stamp: "Formatted at 2025-10-23 02:08:01 on dist-test-slave-bfbt"
I20251023 02:08:01.296559 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:01.315804 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:01.316407 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:01.316978 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:01.318244 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:01.318329 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:01.318440 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:01.318516 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:01.335830 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:38495
I20251023 02:08:01.335883 16987 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:38495 every 8 connection(s)
I20251023 02:08:01.343184 16988 heartbeater.cc:344] Connected to a master server at 127.15.161.254:44077
I20251023 02:08:01.343302 16988 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:01.343712 16988 heartbeater.cc:507] Master 127.15.161.254:44077 requested a full tablet report, sending...
I20251023 02:08:01.344411 16567 ts_manager.cc:194] Registered new tserver with Master: 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196:38495)
I20251023 02:08:01.345155 16567 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40198
I20251023 02:08:01.356695 16874 raft_consensus.cc:493] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:01.356884 16874 raft_consensus.cc:515] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.357470 16874 leader_election.cc:290] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), 246467b191014383b97905412e5ce7cf (127.15.161.194:40047)
I20251023 02:08:01.357851 16670 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0c1212f78ee4862a7df030fab315c99" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f6ec407b1d402f8566ea125c0eb810" is_pre_election: true
I20251023 02:08:01.357883 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0c1212f78ee4862a7df030fab315c99" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246467b191014383b97905412e5ce7cf" is_pre_election: true
I20251023 02:08:01.358165 16670 raft_consensus.cc:2393] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8618ac81e1549178549865eb66bdfb4 in current term 1: Already voted for candidate 246467b191014383b97905412e5ce7cf in this term.
I20251023 02:08:01.358469 16785 leader_election.cc:304] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8618ac81e1549178549865eb66bdfb4; no voters: 246467b191014383b97905412e5ce7cf, 68f6ec407b1d402f8566ea125c0eb810
I20251023 02:08:01.358712 16874 raft_consensus.cc:3060] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:01.360370 16874 raft_consensus.cc:2749] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:08:01.371703 16870 consensus_queue.cc:1048] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.375543 16900 consensus_queue.cc:1048] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.397064 16871 consensus_queue.cc:1048] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.401182 16997 consensus_queue.cc:1048] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.409482 16894 consensus_queue.cc:1048] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.412967 16894 consensus_queue.cc:1048] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.448022 16894 consensus_queue.cc:1048] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.451891 16900 consensus_queue.cc:1048] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.468461 16621 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:01.476092 16745 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab"
dest_uuid: "246467b191014383b97905412e5ce7cf"
mode: GRACEFUL
new_leader_uuid: "68f6ec407b1d402f8566ea125c0eb810"
from {username='slave'} at 127.0.0.1:42540
I20251023 02:08:01.476343 16745 raft_consensus.cc:606] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Received request to transfer leadership to TS 68f6ec407b1d402f8566ea125c0eb810
I20251023 02:08:01.477322 16745 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4"
dest_uuid: "246467b191014383b97905412e5ce7cf"
mode: GRACEFUL
new_leader_uuid: "a8618ac81e1549178549865eb66bdfb4"
from {username='slave'} at 127.0.0.1:42540
I20251023 02:08:01.477464 16745 raft_consensus.cc:606] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Received request to transfer leadership to TS a8618ac81e1549178549865eb66bdfb4
I20251023 02:08:01.477810 16621 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251023 02:08:01.478086 16621 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:01.496718 16900 consensus_queue.cc:1048] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.500140 16894 consensus_queue.cc:1048] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.501760 16894 consensus_queue.cc:1048] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.503985 16894 raft_consensus.cc:993] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: : Instructing follower a8618ac81e1549178549865eb66bdfb4 to start an election
I20251023 02:08:01.504148 16894 raft_consensus.cc:1081] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Signalling peer a8618ac81e1549178549865eb66bdfb4 to start an election
I20251023 02:08:01.506295 16870 consensus_queue.cc:1048] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.506404 16817 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4"
dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
from {username='slave'} at 127.0.0.1:41222
I20251023 02:08:01.506631 16817 raft_consensus.cc:493] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:08:01.506757 16817 raft_consensus.cc:3060] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.508955 16817 raft_consensus.cc:515] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.510227 16817 leader_election.cc:290] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 2 election: Requested vote from peers 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193:42593), 246467b191014383b97905412e5ce7cf (127.15.161.194:40047)
I20251023 02:08:01.510447 16669 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
I20251023 02:08:01.510519 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d59c893ae0747279b76ee40c85f2ec4" candidate_uuid: "a8618ac81e1549178549865eb66bdfb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "246467b191014383b97905412e5ce7cf"
I20251023 02:08:01.510691 16745 raft_consensus.cc:3055] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:08:01.510697 16669 raft_consensus.cc:3060] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.510802 16745 raft_consensus.cc:740] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:01.511034 16745 consensus_queue.cc:260] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.511297 16745 raft_consensus.cc:3060] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.513063 16745 raft_consensus.cc:2468] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8618ac81e1549178549865eb66bdfb4 in term 2.
I20251023 02:08:01.513415 16784 leader_election.cc:304] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246467b191014383b97905412e5ce7cf, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:01.513652 16874 raft_consensus.cc:2804] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:08:01.513782 16874 raft_consensus.cc:697] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 2 LEADER]: Becoming Leader. State: Replica: a8618ac81e1549178549865eb66bdfb4, State: Running, Role: LEADER
I20251023 02:08:01.514010 16874 consensus_queue.cc:237] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.515517 16669 raft_consensus.cc:2468] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8618ac81e1549178549865eb66bdfb4 in term 2.
I20251023 02:08:01.516073 16566 catalog_manager.cc:5649] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 reported cstate change: term changed from 1 to 2, leader changed from 246467b191014383b97905412e5ce7cf (127.15.161.194) to a8618ac81e1549178549865eb66bdfb4 (127.15.161.195). New cstate: current_term: 2 leader_uuid: "a8618ac81e1549178549865eb66bdfb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: HEALTHY } } }
I20251023 02:08:01.659592 16874 consensus_queue.cc:1048] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.663609 17009 consensus_queue.cc:1048] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.861459 16817 raft_consensus.cc:1275] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Refusing update from remote peer 68f6ec407b1d402f8566ea125c0eb810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:08:01.861944 16997 consensus_queue.cc:1048] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:01.865833 16745 raft_consensus.cc:1275] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Refusing update from remote peer 68f6ec407b1d402f8566ea125c0eb810: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:08:01.866600 16907 consensus_queue.cc:1048] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:01.903043 16745 raft_consensus.cc:1275] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Refusing update from remote peer a8618ac81e1549178549865eb66bdfb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:08:01.903551 17009 consensus_queue.cc:1048] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:01.906713 16670 raft_consensus.cc:1275] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Refusing update from remote peer a8618ac81e1549178549865eb66bdfb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:08:01.907202 16913 consensus_queue.cc:1048] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:01.988520 16900 raft_consensus.cc:993] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: : Instructing follower 68f6ec407b1d402f8566ea125c0eb810 to start an election
I20251023 02:08:01.988734 16900 raft_consensus.cc:1081] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Signalling peer 68f6ec407b1d402f8566ea125c0eb810 to start an election
I20251023 02:08:01.989295 16670 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab"
dest_uuid: "68f6ec407b1d402f8566ea125c0eb810"
from {username='slave'} at 127.0.0.1:39734
I20251023 02:08:01.989456 16670 raft_consensus.cc:493] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:08:01.989567 16670 raft_consensus.cc:3060] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.991300 16670 raft_consensus.cc:515] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.991878 16670 leader_election.cc:290] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 2 election: Requested vote from peers 246467b191014383b97905412e5ce7cf (127.15.161.194:40047), a8618ac81e1549178549865eb66bdfb4 (127.15.161.195:32909)
I20251023 02:08:01.992250 16818 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8618ac81e1549178549865eb66bdfb4"
I20251023 02:08:01.992453 16818 raft_consensus.cc:3060] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.992745 16745 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a631d579ece4a1fa4a7210a36a992ab" candidate_uuid: "68f6ec407b1d402f8566ea125c0eb810" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "246467b191014383b97905412e5ce7cf"
I20251023 02:08:01.992908 16745 raft_consensus.cc:3055] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:08:01.993005 16745 raft_consensus.cc:740] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 246467b191014383b97905412e5ce7cf, State: Running, Role: LEADER
I20251023 02:08:01.993243 16745 consensus_queue.cc:260] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.993553 16745 raft_consensus.cc:3060] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:08:01.995077 16818 raft_consensus.cc:2468] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 2.
I20251023 02:08:01.995467 16633 leader_election.cc:304] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f6ec407b1d402f8566ea125c0eb810, a8618ac81e1549178549865eb66bdfb4; no voters:
I20251023 02:08:01.995739 16907 raft_consensus.cc:2804] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:08:01.995901 16745 raft_consensus.cc:2468] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f6ec407b1d402f8566ea125c0eb810 in term 2.
I20251023 02:08:01.995905 16907 raft_consensus.cc:697] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 2 LEADER]: Becoming Leader. State: Replica: 68f6ec407b1d402f8566ea125c0eb810, State: Running, Role: LEADER
I20251023 02:08:01.996253 16907 consensus_queue.cc:237] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } }
I20251023 02:08:01.998288 16568 catalog_manager.cc:5649] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 reported cstate change: term changed from 1 to 2, leader changed from 246467b191014383b97905412e5ce7cf (127.15.161.194) to 68f6ec407b1d402f8566ea125c0eb810 (127.15.161.193). New cstate: current_term: 2 leader_uuid: "68f6ec407b1d402f8566ea125c0eb810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:02.346167 16988 heartbeater.cc:499] Master 127.15.161.254:44077 was elected leader, sending a full tablet report...
I20251023 02:08:02.409830 16818 raft_consensus.cc:1275] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Refusing update from remote peer 68f6ec407b1d402f8566ea125c0eb810: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:08:02.410351 16907 consensus_queue.cc:1048] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:02.413187 16745 raft_consensus.cc:1275] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Refusing update from remote peer 68f6ec407b1d402f8566ea125c0eb810: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:08:02.413731 16871 consensus_queue.cc:1048] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:03.478612 16621 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:03.479758 16621 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:03.479907 16621 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:05.480295 16621 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:05.481673 16621 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:05.481834 16621 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:06.482971 16817 consensus_queue.cc:237] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } }
I20251023 02:08:06.487857 16745 raft_consensus.cc:1275] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Refusing update from remote peer a8618ac81e1549178549865eb66bdfb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:08:06.487874 16669 raft_consensus.cc:1275] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Refusing update from remote peer a8618ac81e1549178549865eb66bdfb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:08:06.488444 17018 consensus_queue.cc:1048] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:06.488735 17022 consensus_queue.cc:1048] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:06.492417 17018 raft_consensus.cc:2955] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.493129 16745 raft_consensus.cc:2955] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.493129 16669 raft_consensus.cc:2955] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.497720 16566 catalog_manager.cc:5649] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 reported cstate change: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New cstate: current_term: 2 leader_uuid: "a8618ac81e1549178549865eb66bdfb4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 02:08:06.504825 16783 consensus_peers.cc:597] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 -> Peer 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196:38495): Couldn't send request to peer 0ba79ef4fac849aa911b2f4679b8120d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d59c893ae0747279b76ee40c85f2ec4. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:08:06.511158 16669 consensus_queue.cc:237] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } attrs { replace: true } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } }
I20251023 02:08:06.514858 16745 raft_consensus.cc:1275] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Refusing update from remote peer 68f6ec407b1d402f8566ea125c0eb810: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:08:06.515627 16907 consensus_queue.cc:1048] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:06.518996 16817 raft_consensus.cc:1275] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Refusing update from remote peer 68f6ec407b1d402f8566ea125c0eb810: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:08:06.519479 16907 raft_consensus.cc:2955] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } attrs { replace: true } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.519841 16745 raft_consensus.cc:2955] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } attrs { replace: true } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
W20251023 02:08:06.521411 16634 consensus_peers.cc:597] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 -> Peer 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196:38495): Couldn't send request to peer 0ba79ef4fac849aa911b2f4679b8120d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3a8c5297ca846d7acd98685392ac164. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:08:06.523615 17037 consensus_queue.cc:1048] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:06.525174 16817 consensus_queue.cc:237] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } attrs { replace: true } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } }
I20251023 02:08:06.525125 16567 catalog_manager.cc:5649] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf reported cstate change: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New cstate: current_term: 2 leader_uuid: "68f6ec407b1d402f8566ea125c0eb810" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } attrs { replace: true } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.525528 16818 raft_consensus.cc:2955] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } attrs { replace: true } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.530839 16669 raft_consensus.cc:1275] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Refusing update from remote peer a8618ac81e1549178549865eb66bdfb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:06.531049 16745 raft_consensus.cc:1275] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Refusing update from remote peer a8618ac81e1549178549865eb66bdfb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 02:08:06.531116 16783 consensus_peers.cc:597] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 -> Peer 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196:38495): Couldn't send request to peer 0ba79ef4fac849aa911b2f4679b8120d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d481d3550dff46dd95f16a66d80ddfe5. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:08:06.531599 17027 consensus_queue.cc:1048] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:06.531885 17022 consensus_queue.cc:1048] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:06.534912 17027 raft_consensus.cc:2955] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } attrs { replace: true } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.535737 16669 raft_consensus.cc:2955] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } attrs { replace: true } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.536048 16745 raft_consensus.cc:2955] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } attrs { replace: true } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.541473 16567 catalog_manager.cc:5649] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf reported cstate change: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "a8618ac81e1549178549865eb66bdfb4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } attrs { replace: true } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.548938 16745 consensus_queue.cc:237] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } }
I20251023 02:08:06.551334 16669 raft_consensus.cc:1275] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Refusing update from remote peer 246467b191014383b97905412e5ce7cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:06.551635 16817 raft_consensus.cc:1275] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Refusing update from remote peer 246467b191014383b97905412e5ce7cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:06.551788 17045 consensus_queue.cc:1048] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:06.552208 16900 consensus_queue.cc:1048] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:06.554868 17045 raft_consensus.cc:2955] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.555949 16669 raft_consensus.cc:2955] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
I20251023 02:08:06.560479 16566 catalog_manager.cc:5649] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf reported cstate change: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "246467b191014383b97905412e5ce7cf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:06.560611 16818 raft_consensus.cc:2955] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "68f6ec407b1d402f8566ea125c0eb810" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 42593 } attrs { replace: true } } peers { permanent_uuid: "246467b191014383b97905412e5ce7cf" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 40047 } } peers { permanent_uuid: "a8618ac81e1549178549865eb66bdfb4" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 32909 } } peers { permanent_uuid: "0ba79ef4fac849aa911b2f4679b8120d" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 38495 } attrs { promote: true } } }
W20251023 02:08:06.562398 16709 consensus_peers.cc:597] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf -> Peer 0ba79ef4fac849aa911b2f4679b8120d (127.15.161.196:38495): Couldn't send request to peer 0ba79ef4fac849aa911b2f4679b8120d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0c1212f78ee4862a7df030fab315c99. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:08:06.595113 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
W20251023 02:08:06.599316 16555 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20251023 02:08:06.601231 16616 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:08:06.601346 16616 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 02:08:06.604035 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:06.604343 16007 tablet_replica.cc:333] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.604539 16007 raft_consensus.cc:2243] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.604835 16007 raft_consensus.cc:2272] T 5d59c893ae0747279b76ee40c85f2ec4 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.609252 16007 tablet_replica.cc:333] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.609457 16007 raft_consensus.cc:2243] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.609647 16007 raft_consensus.cc:2272] T c036f3aca6224ae3b725c72e2aafc162 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.610217 16007 tablet_replica.cc:333] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.610356 16007 raft_consensus.cc:2243] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.610615 16007 raft_consensus.cc:2272] T d481d3550dff46dd95f16a66d80ddfe5 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.611220 16007 tablet_replica.cc:333] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.611366 16007 raft_consensus.cc:2243] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.611542 16007 raft_consensus.cc:2272] T 7af85c4409a641d593757e92cb737877 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.612069 16007 tablet_replica.cc:333] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.612213 16007 raft_consensus.cc:2243] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.612571 16007 raft_consensus.cc:2272] T eae456735a20434a862c20ccea930c4b P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.613096 16007 tablet_replica.cc:333] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.613263 16007 raft_consensus.cc:2243] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.613498 16007 raft_consensus.cc:2272] T 2a631d579ece4a1fa4a7210a36a992ab P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.613971 16007 tablet_replica.cc:333] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.614110 16007 raft_consensus.cc:2243] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.614507 16007 raft_consensus.cc:2272] T c3a8c5297ca846d7acd98685392ac164 P 68f6ec407b1d402f8566ea125c0eb810 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.615131 16007 tablet_replica.cc:333] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810: stopping tablet replica
I20251023 02:08:06.615316 16007 raft_consensus.cc:2243] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.615489 16007 raft_consensus.cc:2272] T d0c1212f78ee4862a7df030fab315c99 P 68f6ec407b1d402f8566ea125c0eb810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.621836 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:08:06.626621 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
W20251023 02:08:06.628232 16616 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:08:06.628369 16616 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 02:08:06.635504 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:06.635771 16007 tablet_replica.cc:333] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.635998 16007 raft_consensus.cc:2243] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.636253 16007 raft_consensus.cc:2272] T 5d59c893ae0747279b76ee40c85f2ec4 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.637117 16007 tablet_replica.cc:333] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.637300 16007 raft_consensus.cc:2243] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.637501 16007 raft_consensus.cc:2272] T d481d3550dff46dd95f16a66d80ddfe5 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.638212 16007 tablet_replica.cc:333] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.638399 16007 raft_consensus.cc:2243] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.638700 16007 raft_consensus.cc:2272] T 7af85c4409a641d593757e92cb737877 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.639238 16007 tablet_replica.cc:333] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.639367 16007 raft_consensus.cc:2243] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.639675 16007 raft_consensus.cc:2272] T c036f3aca6224ae3b725c72e2aafc162 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.640172 16007 tablet_replica.cc:333] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.640318 16007 raft_consensus.cc:2243] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.640525 16007 raft_consensus.cc:2272] T c3a8c5297ca846d7acd98685392ac164 P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.641211 16007 tablet_replica.cc:333] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.641360 16007 raft_consensus.cc:2243] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.641698 16007 raft_consensus.cc:2272] T d0c1212f78ee4862a7df030fab315c99 P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.642354 16007 tablet_replica.cc:333] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.642493 16007 raft_consensus.cc:2243] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.642611 16007 raft_consensus.cc:2272] T eae456735a20434a862c20ccea930c4b P 246467b191014383b97905412e5ce7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.643066 16007 tablet_replica.cc:333] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf: stopping tablet replica
I20251023 02:08:06.643168 16007 raft_consensus.cc:2243] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.643357 16007 raft_consensus.cc:2272] T 2a631d579ece4a1fa4a7210a36a992ab P 246467b191014383b97905412e5ce7cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.659610 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:08:06.663735 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
W20251023 02:08:06.667074 16616 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:08:06.667186 16616 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:08:06.669045 16616 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:08:06.669139 16616 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 02:08:06.672535 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:06.672873 16007 tablet_replica.cc:333] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.673040 16007 raft_consensus.cc:2243] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.673213 16007 raft_consensus.cc:2272] T c036f3aca6224ae3b725c72e2aafc162 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.673722 16007 tablet_replica.cc:333] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.673843 16007 raft_consensus.cc:2243] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.674116 16007 raft_consensus.cc:2272] T 5d59c893ae0747279b76ee40c85f2ec4 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.674800 16007 tablet_replica.cc:333] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.674919 16007 raft_consensus.cc:2243] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.675063 16007 raft_consensus.cc:2272] T d0c1212f78ee4862a7df030fab315c99 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.675555 16007 tablet_replica.cc:333] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.675695 16007 raft_consensus.cc:2243] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.675822 16007 raft_consensus.cc:2272] T c3a8c5297ca846d7acd98685392ac164 P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.676321 16007 tablet_replica.cc:333] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.676429 16007 raft_consensus.cc:2243] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.676550 16007 raft_consensus.cc:2272] T eae456735a20434a862c20ccea930c4b P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.676913 16007 tablet_replica.cc:333] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.677028 16007 raft_consensus.cc:2243] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.677145 16007 raft_consensus.cc:2272] T 7af85c4409a641d593757e92cb737877 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.677493 16007 tablet_replica.cc:333] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.677587 16007 raft_consensus.cc:2243] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.677870 16007 raft_consensus.cc:2272] T d481d3550dff46dd95f16a66d80ddfe5 P a8618ac81e1549178549865eb66bdfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.678401 16007 tablet_replica.cc:333] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4: stopping tablet replica
I20251023 02:08:06.678516 16007 raft_consensus.cc:2243] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:06.678622 16007 raft_consensus.cc:2272] T 2a631d579ece4a1fa4a7210a36a992ab P a8618ac81e1549178549865eb66bdfb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.686372 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:08:06.690546 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:08:06.697856 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:06.710649 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:08:06.713231 16007 master.cc:561] Master@127.15.161.254:44077 shutting down...
I20251023 02:08:06.720860 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:06.721093 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:06.721208 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5373aaa178d24db5bf9aaeb209b066f4: stopping tablet replica
I20251023 02:08:06.735044 16007 master.cc:583] Master@127.15.161.254:44077 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn (7347 ms)
[ RUN ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing
I20251023 02:08:06.756435 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:42585,127.15.161.253:34529,127.15.161.252:39023
I20251023 02:08:06.757253 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:06.760090 17052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:06.760138 17053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:06.761601 17055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:06.761971 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:06.762333 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:06.762418 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:06.762492 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185286762460 us; error 0 us; skew 500 ppm
I20251023 02:08:06.762718 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:06.764307 16007 webserver.cc:492] Webserver started at http://127.15.161.254:36339/ using document root <none> and password file <none>
I20251023 02:08:06.764658 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:06.764777 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:06.764935 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:06.765738 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "eb0fc02b77a742e2b846340de60d538a"
format_stamp: "Formatted at 2025-10-23 02:08:06 on dist-test-slave-bfbt"
I20251023 02:08:06.769407 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251023 02:08:06.772142 17060 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:06.772543 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 02:08:06.772728 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "eb0fc02b77a742e2b846340de60d538a"
format_stamp: "Formatted at 2025-10-23 02:08:06 on dist-test-slave-bfbt"
I20251023 02:08:06.772943 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:06.785171 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:06.785808 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:06.799337 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:42585
I20251023 02:08:06.799368 17111 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:42585 every 8 connection(s)
I20251023 02:08:06.800665 17112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:06.800760 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:06.803830 17114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:06.803862 17115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:06.804245 17112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } has no permanent_uuid. Determining permanent_uuid...
W20251023 02:08:06.804811 17117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:06.807422 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:06.807916 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:06.808408 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:06.808501 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185286808455 us; error 0 us; skew 500 ppm
I20251023 02:08:06.808707 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:06.810091 16007 webserver.cc:492] Webserver started at http://127.15.161.253:45071/ using document root <none> and password file <none>
I20251023 02:08:06.810374 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:06.810494 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:06.810631 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:06.811316 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root/instance:
uuid: "a629b20ca0af4ffdaee78d4b55a4ea65"
format_stamp: "Formatted at 2025-10-23 02:08:06 on dist-test-slave-bfbt"
I20251023 02:08:06.813319 17112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:08:06.814201 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
W20251023 02:08:06.815414 17112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.253:34529: Network error: Client connection negotiation failed: client connection to 127.15.161.253:34529: connect: Connection refused (error 111)
I20251023 02:08:06.816426 17125 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:06.816776 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 02:08:06.816879 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root
uuid: "a629b20ca0af4ffdaee78d4b55a4ea65"
format_stamp: "Formatted at 2025-10-23 02:08:06 on dist-test-slave-bfbt"
I20251023 02:08:06.817014 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:06.829836 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:06.830528 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:06.844098 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.253:34529
I20251023 02:08:06.844136 17176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.253:34529 every 8 connection(s)
I20251023 02:08:06.845450 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:08:06.845499 17177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251023 02:08:06.848153 17180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:06.848454 17179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:06.848944 17177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } has no permanent_uuid. Determining permanent_uuid...
W20251023 02:08:06.850369 17182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:06.850963 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:06.851303 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:06.851394 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:06.851480 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185286851435 us; error 0 us; skew 500 ppm
I20251023 02:08:06.851707 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:06.853055 16007 webserver.cc:492] Webserver started at http://127.15.161.252:42347/ using document root <none> and password file <none>
I20251023 02:08:06.853303 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:06.853408 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:06.853536 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:06.854383 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root/instance:
uuid: "2ec89be5e31944feb1edbd2d95dbe0f0"
format_stamp: "Formatted at 2025-10-23 02:08:06 on dist-test-slave-bfbt"
I20251023 02:08:06.856621 17177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:08:06.857434 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251023 02:08:06.859921 17189 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:06.861357 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 02:08:06.861495 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root
uuid: "2ec89be5e31944feb1edbd2d95dbe0f0"
format_stamp: "Formatted at 2025-10-23 02:08:06 on dist-test-slave-bfbt"
I20251023 02:08:06.861654 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:06.864660 17177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
W20251023 02:08:06.866649 17177 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.252:39023: Network error: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:06.871647 17112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } attempt: 1
I20251023 02:08:06.877533 17112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
W20251023 02:08:06.879046 17112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.252:39023: Network error: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:06.898324 17112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } attempt: 1
W20251023 02:08:06.900054 17112 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.252:39023: Network error: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:06.929901 17177 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } attempt: 1
W20251023 02:08:06.931701 17177 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.161.252:39023: Network error: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:06.932200 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:06.932797 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:06.945484 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.252:39023
I20251023 02:08:06.945523 17241 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.252:39023 every 8 connection(s)
I20251023 02:08:06.946466 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:08:06.946943 17242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:06.949640 17242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:08:06.956097 17242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:08:06.961443 17242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
I20251023 02:08:06.966200 17112 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } attempt: 2
I20251023 02:08:06.970036 17242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0: Bootstrap starting.
I20251023 02:08:06.971724 17242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:06.973527 17242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0: No bootstrap required, opened a new log
I20251023 02:08:06.974326 17242 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:06.974408 17112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: Bootstrap starting.
I20251023 02:08:06.974524 17242 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:06.974618 17242 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec89be5e31944feb1edbd2d95dbe0f0, State: Initialized, Role: FOLLOWER
I20251023 02:08:06.974864 17242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:06.975548 17247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:06.975685 17242 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:08:06.975795 17247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:08:06.976159 17112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:06.983145 17112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: No bootstrap required, opened a new log
I20251023 02:08:06.983943 17177 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } attempt: 2
I20251023 02:08:06.983963 17112 raft_consensus.cc:359] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:06.984144 17112 raft_consensus.cc:385] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:06.984225 17112 raft_consensus.cc:740] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0fc02b77a742e2b846340de60d538a, State: Initialized, Role: FOLLOWER
I20251023 02:08:06.984472 17112 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:06.985133 17258 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:06.985407 17112 sys_catalog.cc:565] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:08:06.985402 17258 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: This master's current role is: FOLLOWER
W20251023 02:08:06.987607 17260 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:08:06.987737 17260 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 02:08:06.991202 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251023 02:08:06.992153 17271 catalog_manager.cc:1568] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:08:06.992242 17271 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 02:08:06.993602 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: Bootstrap starting.
I20251023 02:08:06.994877 17177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:06.996436 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: No bootstrap required, opened a new log
I20251023 02:08:06.997063 17177 raft_consensus.cc:359] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:06.997253 17177 raft_consensus.cc:385] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:06.997324 17177 raft_consensus.cc:740] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a629b20ca0af4ffdaee78d4b55a4ea65, State: Initialized, Role: FOLLOWER
I20251023 02:08:06.997552 17177 consensus_queue.cc:260] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:06.998080 17273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:06.998261 17273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:08:06.998318 17177 sys_catalog.cc:565] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:08:07.001997 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251023 02:08:07.002767 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:07.002905 17284 catalog_manager.cc:1568] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:08:07.002996 17284 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251023 02:08:07.005903 17285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:07.005939 17286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:07.006870 17288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:07.007313 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:07.007598 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:07.007692 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:07.007751 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185287007731 us; error 0 us; skew 500 ppm
I20251023 02:08:07.007920 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:07.009123 16007 webserver.cc:492] Webserver started at http://127.15.161.193:44337/ using document root <none> and password file <none>
I20251023 02:08:07.009364 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:07.009442 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:07.009584 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:07.010234 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "4e0d00f9837c4e1db169ba073af29c87"
format_stamp: "Formatted at 2025-10-23 02:08:07 on dist-test-slave-bfbt"
I20251023 02:08:07.012590 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 02:08:07.014207 17293 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.014534 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:08:07.014649 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "4e0d00f9837c4e1db169ba073af29c87"
format_stamp: "Formatted at 2025-10-23 02:08:07 on dist-test-slave-bfbt"
I20251023 02:08:07.014761 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:07.037115 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:07.037752 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:07.038345 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:07.039254 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:07.039343 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.039449 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:07.039523 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.056435 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:40423
I20251023 02:08:07.056481 17355 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:40423 every 8 connection(s)
I20251023 02:08:07.070369 17356 heartbeater.cc:344] Connected to a master server at 127.15.161.254:42585
I20251023 02:08:07.070581 17356 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.071230 17356 heartbeater.cc:507] Master 127.15.161.254:42585 requested a full tablet report, sending...
I20251023 02:08:07.072490 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:08:07.072934 17077 ts_manager.cc:194] Registered new tserver with Master: 4e0d00f9837c4e1db169ba073af29c87 (127.15.161.193:40423)
I20251023 02:08:07.074155 17357 heartbeater.cc:344] Connected to a master server at 127.15.161.252:39023
I20251023 02:08:07.074335 17357 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.074823 17357 heartbeater.cc:507] Master 127.15.161.252:39023 requested a full tablet report, sending...
I20251023 02:08:07.076138 17207 ts_manager.cc:194] Registered new tserver with Master: 4e0d00f9837c4e1db169ba073af29c87 (127.15.161.193:40423)
W20251023 02:08:07.078428 17364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:07.078455 17363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:07.079228 17358 heartbeater.cc:344] Connected to a master server at 127.15.161.253:34529
I20251023 02:08:07.079380 17358 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.079809 17358 heartbeater.cc:507] Master 127.15.161.253:34529 requested a full tablet report, sending...
W20251023 02:08:07.079918 17366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:07.080789 17142 ts_manager.cc:194] Registered new tserver with Master: 4e0d00f9837c4e1db169ba073af29c87 (127.15.161.193:40423)
I20251023 02:08:07.081054 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:07.081348 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:07.081415 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:07.081489 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185287081457 us; error 0 us; skew 500 ppm
I20251023 02:08:07.081656 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:07.082878 16007 webserver.cc:492] Webserver started at http://127.15.161.194:45613/ using document root <none> and password file <none>
I20251023 02:08:07.083099 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:07.083190 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:07.083320 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:07.084030 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "d63b4544868a4b9bb9828f6128454b07"
format_stamp: "Formatted at 2025-10-23 02:08:07 on dist-test-slave-bfbt"
I20251023 02:08:07.086620 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:08:07.088748 17371 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.089087 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251023 02:08:07.089188 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "d63b4544868a4b9bb9828f6128454b07"
format_stamp: "Formatted at 2025-10-23 02:08:07 on dist-test-slave-bfbt"
I20251023 02:08:07.089320 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:07.117532 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:07.118252 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:07.118875 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:07.122714 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:07.122793 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.123391 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:07.123459 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251023 02:08:07.140261 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:43355
I20251023 02:08:07.140300 17433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:43355 every 8 connection(s)
I20251023 02:08:07.143831 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:08:07.152762 17434 heartbeater.cc:344] Connected to a master server at 127.15.161.254:42585
I20251023 02:08:07.152952 17434 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.153146 17435 heartbeater.cc:344] Connected to a master server at 127.15.161.252:39023
I20251023 02:08:07.153367 17435 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.153527 17434 heartbeater.cc:507] Master 127.15.161.254:42585 requested a full tablet report, sending...
I20251023 02:08:07.153942 17435 heartbeater.cc:507] Master 127.15.161.252:39023 requested a full tablet report, sending...
I20251023 02:08:07.154647 17077 ts_manager.cc:194] Registered new tserver with Master: d63b4544868a4b9bb9828f6128454b07 (127.15.161.194:43355)
W20251023 02:08:07.154983 17441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:07.155242 17207 ts_manager.cc:194] Registered new tserver with Master: d63b4544868a4b9bb9828f6128454b07 (127.15.161.194:43355)
W20251023 02:08:07.156548 17442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:07.158092 17444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:07.158154 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:07.158504 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:07.158623 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:07.158697 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185287158672 us; error 0 us; skew 500 ppm
I20251023 02:08:07.158901 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:07.160006 17436 heartbeater.cc:344] Connected to a master server at 127.15.161.253:34529
I20251023 02:08:07.160171 17436 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.160243 16007 webserver.cc:492] Webserver started at http://127.15.161.195:36893/ using document root <none> and password file <none>
I20251023 02:08:07.160483 17436 heartbeater.cc:507] Master 127.15.161.253:34529 requested a full tablet report, sending...
I20251023 02:08:07.160502 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:07.160600 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:07.160756 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:07.161459 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "891da720cd8a4ae384352e034899aac6"
format_stamp: "Formatted at 2025-10-23 02:08:07 on dist-test-slave-bfbt"
I20251023 02:08:07.161495 17142 ts_manager.cc:194] Registered new tserver with Master: d63b4544868a4b9bb9828f6128454b07 (127.15.161.194:43355)
I20251023 02:08:07.164103 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:08:07.165637 17449 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.165962 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 02:08:07.166095 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "891da720cd8a4ae384352e034899aac6"
format_stamp: "Formatted at 2025-10-23 02:08:07 on dist-test-slave-bfbt"
I20251023 02:08:07.166251 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:07.195056 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:07.195667 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:07.196339 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:07.197441 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:07.197530 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.197624 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:07.197700 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:07.217473 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:34615
I20251023 02:08:07.217492 17511 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:34615 every 8 connection(s)
I20251023 02:08:07.232354 17512 heartbeater.cc:344] Connected to a master server at 127.15.161.254:42585
I20251023 02:08:07.232542 17512 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.232944 17512 heartbeater.cc:507] Master 127.15.161.254:42585 requested a full tablet report, sending...
I20251023 02:08:07.234094 17077 ts_manager.cc:194] Registered new tserver with Master: 891da720cd8a4ae384352e034899aac6 (127.15.161.195:34615)
I20251023 02:08:07.238173 17513 heartbeater.cc:344] Connected to a master server at 127.15.161.252:39023
I20251023 02:08:07.238339 17513 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.238845 17513 heartbeater.cc:507] Master 127.15.161.252:39023 requested a full tablet report, sending...
I20251023 02:08:07.239686 17514 heartbeater.cc:344] Connected to a master server at 127.15.161.253:34529
I20251023 02:08:07.239820 17514 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:07.239953 17207 ts_manager.cc:194] Registered new tserver with Master: 891da720cd8a4ae384352e034899aac6 (127.15.161.195:34615)
I20251023 02:08:07.240207 17514 heartbeater.cc:507] Master 127.15.161.253:34529 requested a full tablet report, sending...
I20251023 02:08:07.240934 17142 ts_manager.cc:194] Registered new tserver with Master: 891da720cd8a4ae384352e034899aac6 (127.15.161.195:34615)
I20251023 02:08:07.241600 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.008018615s
I20251023 02:08:07.379472 17247 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:07.379683 17247 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:07.380596 17247 leader_election.cc:290] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0fc02b77a742e2b846340de60d538a (127.15.161.254:42585), a629b20ca0af4ffdaee78d4b55a4ea65 (127.15.161.253:34529)
I20251023 02:08:07.381107 17087 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0fc02b77a742e2b846340de60d538a" is_pre_election: true
I20251023 02:08:07.381294 17152 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" is_pre_election: true
I20251023 02:08:07.381352 17087 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec89be5e31944feb1edbd2d95dbe0f0 in term 0.
I20251023 02:08:07.381469 17152 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec89be5e31944feb1edbd2d95dbe0f0 in term 0.
I20251023 02:08:07.381824 17191 leader_election.cc:304] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec89be5e31944feb1edbd2d95dbe0f0, eb0fc02b77a742e2b846340de60d538a; no voters:
I20251023 02:08:07.382078 17247 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:07.382229 17247 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:07.382332 17247 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:07.423053 17247 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:07.423753 17247 leader_election.cc:290] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [CANDIDATE]: Term 1 election: Requested vote from peers eb0fc02b77a742e2b846340de60d538a (127.15.161.254:42585), a629b20ca0af4ffdaee78d4b55a4ea65 (127.15.161.253:34529)
I20251023 02:08:07.424276 17152 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65"
I20251023 02:08:07.424332 17087 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0fc02b77a742e2b846340de60d538a"
I20251023 02:08:07.424469 17152 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:07.424587 17087 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:07.484139 17152 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec89be5e31944feb1edbd2d95dbe0f0 in term 1.
I20251023 02:08:07.484213 17087 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec89be5e31944feb1edbd2d95dbe0f0 in term 1.
I20251023 02:08:07.484789 17191 leader_election.cc:304] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ec89be5e31944feb1edbd2d95dbe0f0, a629b20ca0af4ffdaee78d4b55a4ea65, eb0fc02b77a742e2b846340de60d538a; no voters:
I20251023 02:08:07.485134 17247 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:07.485499 17247 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec89be5e31944feb1edbd2d95dbe0f0, State: Running, Role: LEADER
I20251023 02:08:07.485733 17247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:07.487255 17520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec89be5e31944feb1edbd2d95dbe0f0. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.487478 17520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:07.488004 17522 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:08:07.489621 17522 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:08:07.492436 17087 raft_consensus.cc:1275] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Refusing update from remote peer 2ec89be5e31944feb1edbd2d95dbe0f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:07.492863 17152 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 1 FOLLOWER]: Refusing update from remote peer 2ec89be5e31944feb1edbd2d95dbe0f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:07.492930 17247 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:07.493340 17520 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:07.495304 17524 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec89be5e31944feb1edbd2d95dbe0f0. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.495481 17524 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:08:07.498245 17247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.498488 17247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:07.499023 17522 catalog_manager.cc:1357] Generated new cluster ID: 830c162f2d1848239cadf91742b6ed6b
I20251023 02:08:07.499112 17522 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:08:07.502804 17524 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.503105 17524 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:08:07.508647 17273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec89be5e31944feb1edbd2d95dbe0f0. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.508877 17273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:08:07.509838 17520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.510041 17520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:07.511234 17273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:07.511448 17273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 02:08:07.515177 17522 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:08:07.516058 17522 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:08:07.528769 17522 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0: Generated new TSK 0
I20251023 02:08:07.529206 17522 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:08:07.993599 17271 catalog_manager.cc:1269] Loaded cluster ID: 830c162f2d1848239cadf91742b6ed6b
I20251023 02:08:07.993738 17271 catalog_manager.cc:1562] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: loading cluster ID for follower catalog manager: success
I20251023 02:08:07.995599 17271 catalog_manager.cc:1584] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: acquiring CA information for follower catalog manager: success
I20251023 02:08:07.996619 17271 catalog_manager.cc:1612] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 02:08:08.004124 17284 catalog_manager.cc:1269] Loaded cluster ID: 830c162f2d1848239cadf91742b6ed6b
I20251023 02:08:08.004209 17284 catalog_manager.cc:1562] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: loading cluster ID for follower catalog manager: success
I20251023 02:08:08.005932 17284 catalog_manager.cc:1584] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: acquiring CA information for follower catalog manager: success
I20251023 02:08:08.007002 17284 catalog_manager.cc:1612] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 02:08:08.078563 17207 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56682
I20251023 02:08:08.157967 17207 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56690
I20251023 02:08:08.242630 17207 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56696
I20251023 02:08:08.986243 17259 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:09.079790 17357 heartbeater.cc:499] Master 127.15.161.252:39023 was elected leader, sending a full tablet report...
I20251023 02:08:09.135008 16007 test_util.cc:276] Using random seed: -410328204
I20251023 02:08:09.159231 17435 heartbeater.cc:499] Master 127.15.161.252:39023 was elected leader, sending a full tablet report...
I20251023 02:08:09.169446 17207 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56704:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251023 02:08:09.207244 17477 tablet_service.cc:1505] Processing CreateTablet for tablet 84f84adb93c44190a27e6120b35095c4 (DEFAULT_TABLE table=test-workload [id=35251810366f47a8ae5cb66e5a1047a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 02:08:09.207727 17399 tablet_service.cc:1505] Processing CreateTablet for tablet 8562c67608cb4f6db5095a9c2a018c72 (DEFAULT_TABLE table=test-workload [id=35251810366f47a8ae5cb66e5a1047a0]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 02:08:09.207926 17477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f84adb93c44190a27e6120b35095c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:09.208216 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8562c67608cb4f6db5095a9c2a018c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:09.213831 17321 tablet_service.cc:1505] Processing CreateTablet for tablet aa2712658508494d9dea95f33a2d4c6d (DEFAULT_TABLE table=test-workload [id=35251810366f47a8ae5cb66e5a1047a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 02:08:09.214449 17321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2712658508494d9dea95f33a2d4c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:09.215222 17561 tablet_bootstrap.cc:492] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07: Bootstrap starting.
I20251023 02:08:09.217710 17561 tablet_bootstrap.cc:654] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:09.218815 17562 tablet_bootstrap.cc:492] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6: Bootstrap starting.
I20251023 02:08:09.222381 17562 tablet_bootstrap.cc:654] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:09.222916 17564 tablet_bootstrap.cc:492] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87: Bootstrap starting.
I20251023 02:08:09.225026 17564 tablet_bootstrap.cc:654] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:09.225373 17561 tablet_bootstrap.cc:492] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07: No bootstrap required, opened a new log
I20251023 02:08:09.225638 17561 ts_tablet_manager.cc:1403] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20251023 02:08:09.231422 17561 raft_consensus.cc:359] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63b4544868a4b9bb9828f6128454b07" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 43355 } }
I20251023 02:08:09.231690 17561 raft_consensus.cc:385] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:09.231822 17561 raft_consensus.cc:740] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63b4544868a4b9bb9828f6128454b07, State: Initialized, Role: FOLLOWER
I20251023 02:08:09.232117 17561 consensus_queue.cc:260] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63b4544868a4b9bb9828f6128454b07" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 43355 } }
I20251023 02:08:09.232416 17561 raft_consensus.cc:399] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:08:09.232451 17564 tablet_bootstrap.cc:492] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87: No bootstrap required, opened a new log
I20251023 02:08:09.232506 17561 raft_consensus.cc:493] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:08:09.232640 17561 raft_consensus.cc:3060] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:09.232620 17564 ts_tablet_manager.cc:1403] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20251023 02:08:09.233244 17562 tablet_bootstrap.cc:492] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6: No bootstrap required, opened a new log
I20251023 02:08:09.233397 17562 ts_tablet_manager.cc:1403] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20251023 02:08:09.233614 17564 raft_consensus.cc:359] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0d00f9837c4e1db169ba073af29c87" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40423 } }
I20251023 02:08:09.233789 17564 raft_consensus.cc:385] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:09.233899 17564 raft_consensus.cc:740] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0d00f9837c4e1db169ba073af29c87, State: Initialized, Role: FOLLOWER
I20251023 02:08:09.234172 17562 raft_consensus.cc:359] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da720cd8a4ae384352e034899aac6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34615 } }
I20251023 02:08:09.234239 17564 consensus_queue.cc:260] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0d00f9837c4e1db169ba073af29c87" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40423 } }
I20251023 02:08:09.234330 17562 raft_consensus.cc:385] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:09.234459 17564 raft_consensus.cc:399] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:08:09.234489 17562 raft_consensus.cc:740] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 891da720cd8a4ae384352e034899aac6, State: Initialized, Role: FOLLOWER
I20251023 02:08:09.234604 17564 raft_consensus.cc:493] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:08:09.234779 17562 consensus_queue.cc:260] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da720cd8a4ae384352e034899aac6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34615 } }
I20251023 02:08:09.235333 17564 raft_consensus.cc:3060] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:09.235364 17562 raft_consensus.cc:399] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:08:09.235562 17562 raft_consensus.cc:493] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:08:09.235720 17562 raft_consensus.cc:3060] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:09.243791 17513 heartbeater.cc:499] Master 127.15.161.252:39023 was elected leader, sending a full tablet report...
I20251023 02:08:09.295152 17561 raft_consensus.cc:515] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63b4544868a4b9bb9828f6128454b07" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 43355 } }
I20251023 02:08:09.295152 17562 raft_consensus.cc:515] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da720cd8a4ae384352e034899aac6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34615 } }
I20251023 02:08:09.295152 17564 raft_consensus.cc:515] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0d00f9837c4e1db169ba073af29c87" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40423 } }
I20251023 02:08:09.295667 17562 leader_election.cc:304] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 891da720cd8a4ae384352e034899aac6; no voters:
I20251023 02:08:09.296017 17564 leader_election.cc:304] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e0d00f9837c4e1db169ba073af29c87; no voters:
I20251023 02:08:09.296159 17561 leader_election.cc:304] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d63b4544868a4b9bb9828f6128454b07; no voters:
I20251023 02:08:09.296940 17562 leader_election.cc:290] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:08:09.297227 17567 raft_consensus.cc:2804] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:09.299592 17564 leader_election.cc:290] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:08:09.299778 17568 raft_consensus.cc:2804] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:09.300942 17562 ts_tablet_manager.cc:1434] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6: Time spent starting tablet: real 0.067s user 0.002s sys 0.006s
I20251023 02:08:09.304483 17567 raft_consensus.cc:697] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 1 LEADER]: Becoming Leader. State: Replica: 891da720cd8a4ae384352e034899aac6, State: Running, Role: LEADER
I20251023 02:08:09.304731 17564 ts_tablet_manager.cc:1434] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87: Time spent starting tablet: real 0.072s user 0.005s sys 0.005s
I20251023 02:08:09.305115 17561 leader_election.cc:290] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:08:09.305017 17567 consensus_queue.cc:237] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da720cd8a4ae384352e034899aac6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34615 } }
I20251023 02:08:09.305040 17569 raft_consensus.cc:2804] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:09.306376 17568 raft_consensus.cc:697] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0d00f9837c4e1db169ba073af29c87, State: Running, Role: LEADER
I20251023 02:08:09.306905 17568 consensus_queue.cc:237] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0d00f9837c4e1db169ba073af29c87" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40423 } }
I20251023 02:08:09.307056 17561 ts_tablet_manager.cc:1434] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07: Time spent starting tablet: real 0.081s user 0.006s sys 0.009s
I20251023 02:08:09.309978 17206 catalog_manager.cc:5649] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0d00f9837c4e1db169ba073af29c87 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "4e0d00f9837c4e1db169ba073af29c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e0d00f9837c4e1db169ba073af29c87" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 40423 } health_report { overall_health: HEALTHY } } }
I20251023 02:08:09.312090 17207 catalog_manager.cc:5649] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 891da720cd8a4ae384352e034899aac6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "891da720cd8a4ae384352e034899aac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da720cd8a4ae384352e034899aac6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34615 } health_report { overall_health: HEALTHY } } }
I20251023 02:08:09.321532 17569 raft_consensus.cc:697] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 1 LEADER]: Becoming Leader. State: Replica: d63b4544868a4b9bb9828f6128454b07, State: Running, Role: LEADER
I20251023 02:08:09.326352 17569 consensus_queue.cc:237] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63b4544868a4b9bb9828f6128454b07" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 43355 } }
I20251023 02:08:09.328773 17206 catalog_manager.cc:5649] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63b4544868a4b9bb9828f6128454b07 (127.15.161.194). New cstate: current_term: 1 leader_uuid: "d63b4544868a4b9bb9828f6128454b07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63b4544868a4b9bb9828f6128454b07" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 43355 } health_report { overall_health: HEALTHY } } }
I20251023 02:08:09.342509 16007 master.cc:561] Master@127.15.161.252:39023 shutting down...
I20251023 02:08:09.349000 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:09.349408 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:09.349536 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2ec89be5e31944feb1edbd2d95dbe0f0: stopping tablet replica
I20251023 02:08:09.354833 16007 master.cc:583] Master@127.15.161.252:39023 shutdown complete.
W20251023 02:08:10.330322 17357 heartbeater.cc:646] Failed to heartbeat to 127.15.161.252:39023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:10.837538 17585 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ec89be5e31944feb1edbd2d95dbe0f0)
I20251023 02:08:10.837759 17585 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:10.838373 17585 leader_election.cc:290] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a629b20ca0af4ffdaee78d4b55a4ea65 (127.15.161.253:34529), 2ec89be5e31944feb1edbd2d95dbe0f0 (127.15.161.252:39023)
I20251023 02:08:10.838877 17152 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0fc02b77a742e2b846340de60d538a" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" is_pre_election: true
I20251023 02:08:10.839067 17152 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0fc02b77a742e2b846340de60d538a in term 1.
I20251023 02:08:10.839710 17064 leader_election.cc:304] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a629b20ca0af4ffdaee78d4b55a4ea65, eb0fc02b77a742e2b846340de60d538a; no voters:
I20251023 02:08:10.839957 17585 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:08:10.840071 17585 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ec89be5e31944feb1edbd2d95dbe0f0)
I20251023 02:08:10.840176 17585 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 1 FOLLOWER]: Advancing to term 2
W20251023 02:08:10.840404 17064 leader_election.cc:336] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ec89be5e31944feb1edbd2d95dbe0f0 (127.15.161.252:39023): Network error: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:10.842002 17585 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:10.842504 17585 leader_election.cc:290] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [CANDIDATE]: Term 2 election: Requested vote from peers a629b20ca0af4ffdaee78d4b55a4ea65 (127.15.161.253:34529), 2ec89be5e31944feb1edbd2d95dbe0f0 (127.15.161.252:39023)
I20251023 02:08:10.842917 17152 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0fc02b77a742e2b846340de60d538a" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65"
I20251023 02:08:10.843087 17152 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 1 FOLLOWER]: Advancing to term 2
W20251023 02:08:10.843727 17064 leader_election.cc:336] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ec89be5e31944feb1edbd2d95dbe0f0 (127.15.161.252:39023): Network error: Client connection negotiation failed: client connection to 127.15.161.252:39023: connect: Connection refused (error 111)
I20251023 02:08:10.844995 17152 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0fc02b77a742e2b846340de60d538a in term 2.
I20251023 02:08:10.845286 17064 leader_election.cc:304] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a629b20ca0af4ffdaee78d4b55a4ea65, eb0fc02b77a742e2b846340de60d538a; no voters: 2ec89be5e31944feb1edbd2d95dbe0f0
I20251023 02:08:10.845530 17585 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:08:10.845813 17585 raft_consensus.cc:697] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 2 LEADER]: Becoming Leader. State: Replica: eb0fc02b77a742e2b846340de60d538a, State: Running, Role: LEADER
I20251023 02:08:10.846029 17585 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } }
I20251023 02:08:10.847144 17588 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0fc02b77a742e2b846340de60d538a. Latest consensus state: current_term: 2 leader_uuid: "eb0fc02b77a742e2b846340de60d538a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb0fc02b77a742e2b846340de60d538a" member_type: VOTER last_known_addr { host: "127.15.161.254" port: 42585 } } peers { permanent_uuid: "a629b20ca0af4ffdaee78d4b55a4ea65" member_type: VOTER last_known_addr { host: "127.15.161.253" port: 34529 } } peers { permanent_uuid: "2ec89be5e31944feb1edbd2d95dbe0f0" member_type: VOTER last_known_addr { host: "127.15.161.252" port: 39023 } } }
I20251023 02:08:10.847353 17588 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:10.847847 17590 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:08:10.849398 17590 catalog_manager.cc:679] Loaded metadata for table test-workload [id=35251810366f47a8ae5cb66e5a1047a0]
I20251023 02:08:10.850728 17590 tablet_loader.cc:96] loaded metadata for tablet 84f84adb93c44190a27e6120b35095c4 (table test-workload [id=35251810366f47a8ae5cb66e5a1047a0])
I20251023 02:08:10.851047 17590 tablet_loader.cc:96] loaded metadata for tablet 8562c67608cb4f6db5095a9c2a018c72 (table test-workload [id=35251810366f47a8ae5cb66e5a1047a0])
I20251023 02:08:10.851336 17590 tablet_loader.cc:96] loaded metadata for tablet aa2712658508494d9dea95f33a2d4c6d (table test-workload [id=35251810366f47a8ae5cb66e5a1047a0])
I20251023 02:08:10.851522 17590 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:08:10.852239 17590 catalog_manager.cc:1269] Loaded cluster ID: 830c162f2d1848239cadf91742b6ed6b
I20251023 02:08:10.852314 17590 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:08:10.853281 17590 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:08:10.853971 17590 catalog_manager.cc:6033] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: Loaded TSK: 0
I20251023 02:08:10.854583 17590 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:08:10.991348 17270 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:10.991493 17270 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:10.996870 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:08:11.004951 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:11.005245 16007 tablet_replica.cc:333] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87: stopping tablet replica
I20251023 02:08:11.005404 16007 raft_consensus.cc:2243] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:11.005554 16007 raft_consensus.cc:2272] T aa2712658508494d9dea95f33a2d4c6d P 4e0d00f9837c4e1db169ba073af29c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:11.018960 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:08:11.021659 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:08:11.029685 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:11.029978 16007 tablet_replica.cc:333] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07: stopping tablet replica
I20251023 02:08:11.030192 16007 raft_consensus.cc:2243] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:11.030356 16007 raft_consensus.cc:2272] T 8562c67608cb4f6db5095a9c2a018c72 P d63b4544868a4b9bb9828f6128454b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:11.043612 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:08:11.046232 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:08:11.053565 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:11.053853 16007 tablet_replica.cc:333] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6: stopping tablet replica
I20251023 02:08:11.054021 16007 raft_consensus.cc:2243] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:11.054216 16007 raft_consensus.cc:2272] T 84f84adb93c44190a27e6120b35095c4 P 891da720cd8a4ae384352e034899aac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:11.067696 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:08:11.070664 16007 master.cc:561] Master@127.15.161.254:42585 shutting down...
I20251023 02:08:11.077492 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:08:11.077811 16007 pending_rounds.cc:70] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: Trying to abort 1 pending ops.
I20251023 02:08:11.078002 16007 pending_rounds.cc:77] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: Aborting op as it isn't in flight: id { term: 2 index: 10 } timestamp: 7213814951307526144 op_type: NO_OP noop_request { }
I20251023 02:08:11.078171 16007 raft_consensus.cc:2889] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:08:11.078297 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:11.078475 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P eb0fc02b77a742e2b846340de60d538a: stopping tablet replica
I20251023 02:08:11.082472 16007 master.cc:583] Master@127.15.161.254:42585 shutdown complete.
I20251023 02:08:11.085968 16007 master.cc:561] Master@127.15.161.253:34529 shutting down...
I20251023 02:08:11.092613 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:11.092845 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:11.092947 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P a629b20ca0af4ffdaee78d4b55a4ea65: stopping tablet replica
I20251023 02:08:11.106542 16007 master.cc:583] Master@127.15.161.253:34529 shutdown complete.
[ OK ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing (4362 ms)
[ RUN ] AutoRebalancerTest.MovesScheduledIfAddTserver
I20251023 02:08:11.118649 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:41203
I20251023 02:08:11.119143 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:11.121526 17591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.121719 17592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:11.123157 16007 server_base.cc:1047] running on GCE node
W20251023 02:08:11.123313 17594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:11.123679 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:11.123754 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:11.123798 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185291123783 us; error 0 us; skew 500 ppm
I20251023 02:08:11.123947 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:11.125061 16007 webserver.cc:492] Webserver started at http://127.15.161.254:33921/ using document root <none> and password file <none>
I20251023 02:08:11.125281 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:11.125342 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:11.125463 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:11.126020 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "605228fd4dc14d699d88c9825181abc4"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.128496 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:11.129964 17599 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.130336 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:11.130438 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "605228fd4dc14d699d88c9825181abc4"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.130549 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:11.136288 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:11.136909 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:11.149317 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:41203
I20251023 02:08:11.149336 17650 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:41203 every 8 connection(s)
I20251023 02:08:11.150627 17651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:11.154247 17651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4: Bootstrap starting.
I20251023 02:08:11.155520 17651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:11.157130 17651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4: No bootstrap required, opened a new log
I20251023 02:08:11.157775 17651 raft_consensus.cc:359] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605228fd4dc14d699d88c9825181abc4" member_type: VOTER }
I20251023 02:08:11.157915 17651 raft_consensus.cc:385] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:11.158006 17651 raft_consensus.cc:740] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605228fd4dc14d699d88c9825181abc4, State: Initialized, Role: FOLLOWER
I20251023 02:08:11.158252 17651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605228fd4dc14d699d88c9825181abc4" member_type: VOTER }
I20251023 02:08:11.158396 17651 raft_consensus.cc:399] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:08:11.158473 17651 raft_consensus.cc:493] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:08:11.158576 17651 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:11.160089 17651 raft_consensus.cc:515] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605228fd4dc14d699d88c9825181abc4" member_type: VOTER }
I20251023 02:08:11.160264 17651 leader_election.cc:304] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 605228fd4dc14d699d88c9825181abc4; no voters:
I20251023 02:08:11.160621 17651 leader_election.cc:290] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:08:11.160779 17654 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:11.161239 17654 raft_consensus.cc:697] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 1 LEADER]: Becoming Leader. State: Replica: 605228fd4dc14d699d88c9825181abc4, State: Running, Role: LEADER
I20251023 02:08:11.161360 17651 sys_catalog.cc:565] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:08:11.161500 17654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605228fd4dc14d699d88c9825181abc4" member_type: VOTER }
I20251023 02:08:11.162467 17656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 605228fd4dc14d699d88c9825181abc4. Latest consensus state: current_term: 1 leader_uuid: "605228fd4dc14d699d88c9825181abc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605228fd4dc14d699d88c9825181abc4" member_type: VOTER } }
I20251023 02:08:11.162442 17655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "605228fd4dc14d699d88c9825181abc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "605228fd4dc14d699d88c9825181abc4" member_type: VOTER } }
I20251023 02:08:11.162695 17656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:11.162716 17655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:11.166863 17659 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:08:11.169143 17659 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:08:11.172667 17659 catalog_manager.cc:1357] Generated new cluster ID: fd2f0dccd9d944339081e1372a2cce09
I20251023 02:08:11.172804 17659 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:08:11.173581 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:08:11.190578 17659 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:08:11.191500 17659 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:08:11.202533 17659 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4: Generated new TSK 0
I20251023 02:08:11.202816 17659 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:08:11.206383 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:11.209450 17673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.209568 17672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.209582 17675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:11.210256 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:11.210520 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:11.210587 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:11.210669 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185291210631 us; error 0 us; skew 500 ppm
I20251023 02:08:11.210824 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:11.211920 16007 webserver.cc:492] Webserver started at http://127.15.161.193:36773/ using document root <none> and password file <none>
I20251023 02:08:11.212149 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:11.212234 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:11.212350 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:11.212949 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "c56bee725057458db50e4cd84930542b"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.215380 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:08:11.216975 17680 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.217365 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 02:08:11.217465 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "c56bee725057458db50e4cd84930542b"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.217583 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:11.228616 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:11.229179 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:11.229679 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:11.230588 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:11.230659 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.230768 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:11.230827 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.243196 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:43999
I20251023 02:08:11.243227 17742 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:43999 every 8 connection(s)
I20251023 02:08:11.244783 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:11.248284 17747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.248705 17748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.251173 17750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:11.251731 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:11.252005 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:11.252066 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:11.252097 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185291252085 us; error 0 us; skew 500 ppm
I20251023 02:08:11.252202 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:11.253311 16007 webserver.cc:492] Webserver started at http://127.15.161.194:33657/ using document root <none> and password file <none>
I20251023 02:08:11.253602 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:11.253712 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:11.253875 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:11.254171 17743 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41203
I20251023 02:08:11.254336 17743 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:11.254637 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "578d6b86d6234a79a4f97d604936681e"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.254729 17743 heartbeater.cc:507] Master 127.15.161.254:41203 requested a full tablet report, sending...
I20251023 02:08:11.255657 17616 ts_manager.cc:194] Registered new tserver with Master: c56bee725057458db50e4cd84930542b (127.15.161.193:43999)
I20251023 02:08:11.256731 17616 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53052
I20251023 02:08:11.257416 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:11.259110 17755 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.259433 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:08:11.259528 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "578d6b86d6234a79a4f97d604936681e"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.259649 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:11.302073 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:11.302752 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:11.303325 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:11.304155 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:11.304242 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.304347 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:11.304422 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.317660 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:42299
I20251023 02:08:11.317693 17817 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:42299 every 8 connection(s)
I20251023 02:08:11.319299 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:11.322924 17822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.323242 17821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:11.324251 17824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:11.327189 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:11.327539 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251023 02:08:11.327592 17818 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41203
W20251023 02:08:11.327641 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:11.327764 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185291327727 us; error 0 us; skew 500 ppm
I20251023 02:08:11.327793 17818 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:11.327942 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:11.328194 17818 heartbeater.cc:507] Master 127.15.161.254:41203 requested a full tablet report, sending...
I20251023 02:08:11.329133 17616 ts_manager.cc:194] Registered new tserver with Master: 578d6b86d6234a79a4f97d604936681e (127.15.161.194:42299)
I20251023 02:08:11.329176 16007 webserver.cc:492] Webserver started at http://127.15.161.195:43517/ using document root <none> and password file <none>
I20251023 02:08:11.329879 17616 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53066
I20251023 02:08:11.337901 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:11.338020 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:11.338176 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:11.338793 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "39cd3cbd74d34c0092b35503bd1e1d66"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.341125 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:08:11.342653 17829 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.343040 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 02:08:11.343137 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "39cd3cbd74d34c0092b35503bd1e1d66"
format_stamp: "Formatted at 2025-10-23 02:08:11 on dist-test-slave-bfbt"
I20251023 02:08:11.343215 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:11.362273 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:11.362746 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:11.363219 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:11.363965 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:11.364019 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.364094 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:11.364132 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:11.377209 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:41463
I20251023 02:08:11.377235 17891 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:41463 every 8 connection(s)
I20251023 02:08:11.383468 17892 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41203
I20251023 02:08:11.383597 17892 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:11.383936 17892 heartbeater.cc:507] Master 127.15.161.254:41203 requested a full tablet report, sending...
I20251023 02:08:11.384732 17616 ts_manager.cc:194] Registered new tserver with Master: 39cd3cbd74d34c0092b35503bd1e1d66 (127.15.161.195:41463)
I20251023 02:08:11.384922 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006769138s
I20251023 02:08:11.385427 17616 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53070
I20251023 02:08:12.258314 17743 heartbeater.cc:499] Master 127.15.161.254:41203 was elected leader, sending a full tablet report...
I20251023 02:08:12.331099 17818 heartbeater.cc:499] Master 127.15.161.254:41203 was elected leader, sending a full tablet report...
I20251023 02:08:12.386615 17892 heartbeater.cc:499] Master 127.15.161.254:41203 was elected leader, sending a full tablet report...
I20251023 02:08:12.413036 16007 test_util.cc:276] Using random seed: -407050139
I20251023 02:08:12.431978 17616 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53072:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:08:12.432539 17616 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:08:12.468972 17707 tablet_service.cc:1505] Processing CreateTablet for tablet dda04d421be546d5a713d9886d8abdfa (DEFAULT_TABLE table=test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251023 02:08:12.468972 17708 tablet_service.cc:1505] Processing CreateTablet for tablet 580961fac15348b3a25d15dbd90a702a (DEFAULT_TABLE table=test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251023 02:08:12.469563 17707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda04d421be546d5a713d9886d8abdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:12.469733 17708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580961fac15348b3a25d15dbd90a702a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:12.475386 17912 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: Bootstrap starting.
I20251023 02:08:12.477061 17912 tablet_bootstrap.cc:654] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:12.479091 17912 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: No bootstrap required, opened a new log
I20251023 02:08:12.479285 17912 ts_tablet_manager.cc:1403] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:08:12.480170 17912 raft_consensus.cc:359] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.480453 17912 raft_consensus.cc:385] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:12.480566 17912 raft_consensus.cc:740] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56bee725057458db50e4cd84930542b, State: Initialized, Role: FOLLOWER
I20251023 02:08:12.480875 17912 consensus_queue.cc:260] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.483747 17912 ts_tablet_manager.cc:1434] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251023 02:08:12.484045 17912 tablet_bootstrap.cc:492] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b: Bootstrap starting.
I20251023 02:08:12.484182 17783 tablet_service.cc:1505] Processing CreateTablet for tablet 580961fac15348b3a25d15dbd90a702a (DEFAULT_TABLE table=test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251023 02:08:12.484671 17783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580961fac15348b3a25d15dbd90a702a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:12.484798 17782 tablet_service.cc:1505] Processing CreateTablet for tablet dda04d421be546d5a713d9886d8abdfa (DEFAULT_TABLE table=test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251023 02:08:12.485208 17782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda04d421be546d5a713d9886d8abdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:12.485924 17912 tablet_bootstrap.cc:654] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:12.489821 17857 tablet_service.cc:1505] Processing CreateTablet for tablet 580961fac15348b3a25d15dbd90a702a (DEFAULT_TABLE table=test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251023 02:08:12.490386 17857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580961fac15348b3a25d15dbd90a702a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:12.492645 17916 tablet_bootstrap.cc:492] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e: Bootstrap starting.
I20251023 02:08:12.494246 17916 tablet_bootstrap.cc:654] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:12.496495 17916 tablet_bootstrap.cc:492] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e: No bootstrap required, opened a new log
I20251023 02:08:12.496379 17856 tablet_service.cc:1505] Processing CreateTablet for tablet dda04d421be546d5a713d9886d8abdfa (DEFAULT_TABLE table=test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251023 02:08:12.496762 17916 ts_tablet_manager.cc:1403] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251023 02:08:12.497046 17856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda04d421be546d5a713d9886d8abdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:12.497581 17916 raft_consensus.cc:359] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.497812 17916 raft_consensus.cc:385] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:12.497892 17916 raft_consensus.cc:740] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578d6b86d6234a79a4f97d604936681e, State: Initialized, Role: FOLLOWER
I20251023 02:08:12.498096 17916 consensus_queue.cc:260] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.499228 17916 ts_tablet_manager.cc:1434] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:08:12.499928 17912 tablet_bootstrap.cc:492] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b: No bootstrap required, opened a new log
I20251023 02:08:12.500105 17912 ts_tablet_manager.cc:1403] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20251023 02:08:12.500859 17912 raft_consensus.cc:359] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.501073 17912 raft_consensus.cc:385] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:12.501156 17912 raft_consensus.cc:740] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56bee725057458db50e4cd84930542b, State: Initialized, Role: FOLLOWER
I20251023 02:08:12.501585 17916 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e: Bootstrap starting.
I20251023 02:08:12.503203 17912 consensus_queue.cc:260] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.503823 17916 tablet_bootstrap.cc:654] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:12.503731 17918 tablet_bootstrap.cc:492] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66: Bootstrap starting.
I20251023 02:08:12.504828 17912 ts_tablet_manager.cc:1434] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251023 02:08:12.506249 17916 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e: No bootstrap required, opened a new log
I20251023 02:08:12.506371 17916 ts_tablet_manager.cc:1403] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251023 02:08:12.506641 17918 tablet_bootstrap.cc:654] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:12.507196 17916 raft_consensus.cc:359] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.507357 17916 raft_consensus.cc:385] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:12.507503 17916 raft_consensus.cc:740] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578d6b86d6234a79a4f97d604936681e, State: Initialized, Role: FOLLOWER
I20251023 02:08:12.507784 17916 consensus_queue.cc:260] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.508213 17916 ts_tablet_manager.cc:1434] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:12.508488 17918 tablet_bootstrap.cc:492] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66: No bootstrap required, opened a new log
I20251023 02:08:12.508603 17918 ts_tablet_manager.cc:1403] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 02:08:12.509141 17918 raft_consensus.cc:359] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.509260 17918 raft_consensus.cc:385] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:12.509325 17918 raft_consensus.cc:740] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39cd3cbd74d34c0092b35503bd1e1d66, State: Initialized, Role: FOLLOWER
I20251023 02:08:12.509506 17918 consensus_queue.cc:260] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.510257 17918 ts_tablet_manager.cc:1434] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:12.510502 17918 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: Bootstrap starting.
I20251023 02:08:12.512400 17914 raft_consensus.cc:493] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:12.512513 17918 tablet_bootstrap.cc:654] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:12.512528 17914 raft_consensus.cc:515] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.513437 17914 leader_election.cc:290] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 578d6b86d6234a79a4f97d604936681e (127.15.161.194:42299), 39cd3cbd74d34c0092b35503bd1e1d66 (127.15.161.195:41463)
I20251023 02:08:12.516227 17918 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: No bootstrap required, opened a new log
I20251023 02:08:12.516398 17918 ts_tablet_manager.cc:1403] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20251023 02:08:12.517459 17918 raft_consensus.cc:359] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.517658 17918 raft_consensus.cc:385] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:12.517776 17918 raft_consensus.cc:740] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39cd3cbd74d34c0092b35503bd1e1d66, State: Initialized, Role: FOLLOWER
I20251023 02:08:12.517992 17918 consensus_queue.cc:260] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.518584 17918 ts_tablet_manager.cc:1434] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:08:12.522040 17793 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "580961fac15348b3a25d15dbd90a702a" candidate_uuid: "c56bee725057458db50e4cd84930542b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578d6b86d6234a79a4f97d604936681e" is_pre_election: true
I20251023 02:08:12.522228 17867 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "580961fac15348b3a25d15dbd90a702a" candidate_uuid: "c56bee725057458db50e4cd84930542b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" is_pre_election: true
I20251023 02:08:12.522333 17793 raft_consensus.cc:2468] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c56bee725057458db50e4cd84930542b in term 0.
I20251023 02:08:12.522485 17867 raft_consensus.cc:2468] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c56bee725057458db50e4cd84930542b in term 0.
I20251023 02:08:12.522951 17684 leader_election.cc:304] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39cd3cbd74d34c0092b35503bd1e1d66, c56bee725057458db50e4cd84930542b; no voters:
I20251023 02:08:12.523185 17914 raft_consensus.cc:2804] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:12.523252 17914 raft_consensus.cc:493] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:12.523305 17914 raft_consensus.cc:3060] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:12.525103 17914 raft_consensus.cc:515] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.525593 17914 leader_election.cc:290] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [CANDIDATE]: Term 1 election: Requested vote from peers 578d6b86d6234a79a4f97d604936681e (127.15.161.194:42299), 39cd3cbd74d34c0092b35503bd1e1d66 (127.15.161.195:41463)
I20251023 02:08:12.525993 17793 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "580961fac15348b3a25d15dbd90a702a" candidate_uuid: "c56bee725057458db50e4cd84930542b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578d6b86d6234a79a4f97d604936681e"
I20251023 02:08:12.526047 17867 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "580961fac15348b3a25d15dbd90a702a" candidate_uuid: "c56bee725057458db50e4cd84930542b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39cd3cbd74d34c0092b35503bd1e1d66"
I20251023 02:08:12.526166 17793 raft_consensus.cc:3060] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:12.526211 17867 raft_consensus.cc:3060] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:12.528115 17793 raft_consensus.cc:2468] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c56bee725057458db50e4cd84930542b in term 1.
I20251023 02:08:12.528116 17867 raft_consensus.cc:2468] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c56bee725057458db50e4cd84930542b in term 1.
I20251023 02:08:12.528501 17684 leader_election.cc:304] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39cd3cbd74d34c0092b35503bd1e1d66, c56bee725057458db50e4cd84930542b; no voters:
I20251023 02:08:12.528715 17914 raft_consensus.cc:2804] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:12.529068 17914 raft_consensus.cc:697] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 1 LEADER]: Becoming Leader. State: Replica: c56bee725057458db50e4cd84930542b, State: Running, Role: LEADER
I20251023 02:08:12.529309 17914 consensus_queue.cc:237] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.531668 17615 catalog_manager.cc:5649] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b reported cstate change: term changed from 0 to 1, leader changed from <none> to c56bee725057458db50e4cd84930542b (127.15.161.193). New cstate: current_term: 1 leader_uuid: "c56bee725057458db50e4cd84930542b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:12.860984 17922 raft_consensus.cc:493] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:08:12.861186 17922 raft_consensus.cc:515] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.862084 17922 leader_election.cc:290] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c56bee725057458db50e4cd84930542b (127.15.161.193:43999), 578d6b86d6234a79a4f97d604936681e (127.15.161.194:42299)
I20251023 02:08:12.868630 17718 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dda04d421be546d5a713d9886d8abdfa" candidate_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c56bee725057458db50e4cd84930542b" is_pre_election: true
I20251023 02:08:12.868863 17793 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dda04d421be546d5a713d9886d8abdfa" candidate_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578d6b86d6234a79a4f97d604936681e" is_pre_election: true
I20251023 02:08:12.868980 17718 raft_consensus.cc:2468] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39cd3cbd74d34c0092b35503bd1e1d66 in term 0.
I20251023 02:08:12.869108 17793 raft_consensus.cc:2468] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39cd3cbd74d34c0092b35503bd1e1d66 in term 0.
I20251023 02:08:12.869419 17832 leader_election.cc:304] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39cd3cbd74d34c0092b35503bd1e1d66, c56bee725057458db50e4cd84930542b; no voters:
I20251023 02:08:12.869666 17922 raft_consensus.cc:2804] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:08:12.869740 17922 raft_consensus.cc:493] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:08:12.869807 17922 raft_consensus.cc:3060] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:12.871532 17922 raft_consensus.cc:515] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.872025 17922 leader_election.cc:290] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [CANDIDATE]: Term 1 election: Requested vote from peers c56bee725057458db50e4cd84930542b (127.15.161.193:43999), 578d6b86d6234a79a4f97d604936681e (127.15.161.194:42299)
I20251023 02:08:12.872481 17718 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dda04d421be546d5a713d9886d8abdfa" candidate_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c56bee725057458db50e4cd84930542b"
I20251023 02:08:12.872489 17793 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dda04d421be546d5a713d9886d8abdfa" candidate_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578d6b86d6234a79a4f97d604936681e"
I20251023 02:08:12.872648 17718 raft_consensus.cc:3060] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:12.872663 17793 raft_consensus.cc:3060] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:12.874883 17718 raft_consensus.cc:2468] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39cd3cbd74d34c0092b35503bd1e1d66 in term 1.
I20251023 02:08:12.874945 17793 raft_consensus.cc:2468] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39cd3cbd74d34c0092b35503bd1e1d66 in term 1.
I20251023 02:08:12.875231 17832 leader_election.cc:304] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39cd3cbd74d34c0092b35503bd1e1d66, c56bee725057458db50e4cd84930542b; no voters:
I20251023 02:08:12.875473 17922 raft_consensus.cc:2804] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:12.875866 17922 raft_consensus.cc:697] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 LEADER]: Becoming Leader. State: Replica: 39cd3cbd74d34c0092b35503bd1e1d66, State: Running, Role: LEADER
I20251023 02:08:12.876075 17922 consensus_queue.cc:237] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } }
I20251023 02:08:12.878429 17615 catalog_manager.cc:5649] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39cd3cbd74d34c0092b35503bd1e1d66 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } health_report { overall_health: HEALTHY } } }
I20251023 02:08:12.997573 17914 consensus_queue.cc:1048] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [LEADER]: Connected to new peer: Peer: permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:13.002362 17914 consensus_queue.cc:1048] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [LEADER]: Connected to new peer: Peer: permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:13.173527 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:13.174011 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:13.174100 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:13.395538 17922 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:13.400053 17938 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:08:15.174530 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:15.175134 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:15.175272 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:17.175783 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:17.176338 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:17.176440 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:19.176810 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:19.177328 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:19.177426 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:21.177904 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:21.178506 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:21.178635 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:23.179091 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:23.179589 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:23.179679 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:23.947918 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:23.950979 17947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:23.951349 17946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:23.951877 17949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:23.953945 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:23.954594 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:23.954671 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:23.954722 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185303954699 us; error 0 us; skew 500 ppm
I20251023 02:08:23.954834 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:23.956030 16007 webserver.cc:492] Webserver started at http://127.15.161.196:34293/ using document root <none> and password file <none>
I20251023 02:08:23.956220 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:23.956314 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:23.956420 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:23.957015 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "ff9a3db794944037963a1cc0d1bf4f20"
format_stamp: "Formatted at 2025-10-23 02:08:23 on dist-test-slave-bfbt"
I20251023 02:08:23.959416 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:08:23.960795 17954 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:23.961119 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:08:23.961230 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "ff9a3db794944037963a1cc0d1bf4f20"
format_stamp: "Formatted at 2025-10-23 02:08:23 on dist-test-slave-bfbt"
I20251023 02:08:23.961357 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:23.977352 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:23.977936 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:23.978507 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:23.979291 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:23.979360 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:23.979468 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:23.979521 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:23.994354 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:39697
I20251023 02:08:23.994370 18016 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:39697 every 8 connection(s)
I20251023 02:08:24.005162 18017 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41203
I20251023 02:08:24.005295 18017 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:24.005599 18017 heartbeater.cc:507] Master 127.15.161.254:41203 requested a full tablet report, sending...
I20251023 02:08:24.006338 17616 ts_manager.cc:194] Registered new tserver with Master: ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196:39697)
I20251023 02:08:24.007014 17616 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35452
I20251023 02:08:24.193591 17867 consensus_queue.cc:237] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } }
I20251023 02:08:24.197378 17718 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:24.197978 17792 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:08:24.198087 18025 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:24.198611 17945 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:08:24.200943 18025 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } } }
I20251023 02:08:24.201705 17718 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } } }
I20251023 02:08:24.205160 17792 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } } }
W20251023 02:08:24.205473 17831 consensus_peers.cc:597] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 -> Peer ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196:39697): Couldn't send request to peer ff9a3db794944037963a1cc0d1bf4f20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dda04d421be546d5a713d9886d8abdfa. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:08:24.206439 17616 catalog_manager.cc:5649] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 reported cstate change: config changed from index -1 to 2, NON_VOTER ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:08:24.763695 18035 ts_tablet_manager.cc:933] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: Initiating tablet copy from peer 39cd3cbd74d34c0092b35503bd1e1d66 (127.15.161.195:41463)
I20251023 02:08:24.764747 18035 tablet_copy_client.cc:323] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:41463
I20251023 02:08:24.770231 17877 tablet_copy_service.cc:140] P 39cd3cbd74d34c0092b35503bd1e1d66: Received BeginTabletCopySession request for tablet dda04d421be546d5a713d9886d8abdfa from peer ff9a3db794944037963a1cc0d1bf4f20 ({username='slave'} at 127.0.0.1:36546)
I20251023 02:08:24.770434 17877 tablet_copy_service.cc:161] P 39cd3cbd74d34c0092b35503bd1e1d66: Beginning new tablet copy session on tablet dda04d421be546d5a713d9886d8abdfa from peer ff9a3db794944037963a1cc0d1bf4f20 at {username='slave'} at 127.0.0.1:36546: session id = ff9a3db794944037963a1cc0d1bf4f20-dda04d421be546d5a713d9886d8abdfa
I20251023 02:08:24.773131 17877 tablet_copy_source_session.cc:215] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:08:24.774746 18035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda04d421be546d5a713d9886d8abdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:24.780239 18035 tablet_copy_client.cc:806] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: tablet copy: Starting download of 0 data blocks...
I20251023 02:08:24.780557 18035 tablet_copy_client.cc:670] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: tablet copy: Starting download of 1 WAL segments...
I20251023 02:08:24.782466 18035 tablet_copy_client.cc:538] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:08:24.785285 18035 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: Bootstrap starting.
I20251023 02:08:24.791405 18035 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:08:24.791777 18035 tablet_bootstrap.cc:492] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: Bootstrap complete.
I20251023 02:08:24.791983 18035 ts_tablet_manager.cc:1403] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: Time spent bootstrapping tablet: real 0.007s user 0.008s sys 0.000s
I20251023 02:08:24.792536 18035 raft_consensus.cc:359] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } }
I20251023 02:08:24.792688 18035 raft_consensus.cc:740] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9a3db794944037963a1cc0d1bf4f20, State: Initialized, Role: LEARNER
I20251023 02:08:24.792852 18035 consensus_queue.cc:260] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: true } }
I20251023 02:08:24.793507 18017 heartbeater.cc:499] Master 127.15.161.254:41203 was elected leader, sending a full tablet report...
I20251023 02:08:24.793531 18035 ts_tablet_manager.cc:1434] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251023 02:08:24.794323 17877 tablet_copy_service.cc:342] P 39cd3cbd74d34c0092b35503bd1e1d66: Request end of tablet copy session ff9a3db794944037963a1cc0d1bf4f20-dda04d421be546d5a713d9886d8abdfa received from {username='slave'} at 127.0.0.1:36546
I20251023 02:08:24.794471 17877 tablet_copy_service.cc:434] P 39cd3cbd74d34c0092b35503bd1e1d66: ending tablet copy session ff9a3db794944037963a1cc0d1bf4f20-dda04d421be546d5a713d9886d8abdfa on tablet dda04d421be546d5a713d9886d8abdfa with peer ff9a3db794944037963a1cc0d1bf4f20
I20251023 02:08:25.179939 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:25.180644 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:25.180763 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:25.380653 17992 raft_consensus.cc:1217] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 02:08:25.897097 18041 raft_consensus.cc:1064] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: attempting to promote NON_VOTER ff9a3db794944037963a1cc0d1bf4f20 to VOTER
I20251023 02:08:25.897912 18041 consensus_queue.cc:237] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } }
I20251023 02:08:25.900432 17992 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 LEARNER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:08:25.900432 17792 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:08:25.900601 17718 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:08:25.900930 18044 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:25.901130 18041 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:25.901371 18046 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:08:25.907169 18046 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.907757 17792 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.908576 17718 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.912159 17614 catalog_manager.cc:5649] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e reported cstate change: config changed from index 2 to 3, ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.913277 17992 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ff9a3db794944037963a1cc0d1bf4f20 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c56bee725057458db50e4cd84930542b" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43999 } attrs { replace: true } } peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.926015 17867 consensus_queue.cc:237] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } }
I20251023 02:08:25.928192 17792 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:08:25.928301 17992 raft_consensus.cc:1275] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 FOLLOWER]: Refusing update from remote peer 39cd3cbd74d34c0092b35503bd1e1d66: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:08:25.928668 18051 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:08:25.928992 18044 consensus_queue.cc:1048] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:08:25.931075 18041 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c56bee725057458db50e4cd84930542b (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.931697 17992 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c56bee725057458db50e4cd84930542b (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.931769 17792 raft_consensus.cc:2955] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c56bee725057458db50e4cd84930542b (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.934983 17615 catalog_manager.cc:5649] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e reported cstate change: config changed from index 3 to 4, VOTER c56bee725057458db50e4cd84930542b (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "578d6b86d6234a79a4f97d604936681e" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 42299 } } peers { permanent_uuid: "39cd3cbd74d34c0092b35503bd1e1d66" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41463 } } peers { permanent_uuid: "ff9a3db794944037963a1cc0d1bf4f20" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 39697 } attrs { promote: false } } }
I20251023 02:08:25.936057 17603 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet dda04d421be546d5a713d9886d8abdfa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:08:25.937880 17708 tablet_service.cc:1552] Processing DeleteTablet for tablet dda04d421be546d5a713d9886d8abdfa with delete_type TABLET_DATA_TOMBSTONED (TS c56bee725057458db50e4cd84930542b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41678
I20251023 02:08:25.938819 18057 tablet_replica.cc:333] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: stopping tablet replica
I20251023 02:08:25.938989 18057 raft_consensus.cc:2243] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:25.939232 18057 raft_consensus.cc:2272] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:25.939942 18057 ts_tablet_manager.cc:1916] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:08:25.943410 18057 ts_tablet_manager.cc:1929] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251023 02:08:25.943513 18057 log.cc:1199] T dda04d421be546d5a713d9886d8abdfa P c56bee725057458db50e4cd84930542b: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/dda04d421be546d5a713d9886d8abdfa
I20251023 02:08:25.944116 17602 catalog_manager.cc:4985] TS c56bee725057458db50e4cd84930542b (127.15.161.193:43999): tablet dda04d421be546d5a713d9886d8abdfa (table test-workload [id=4ea89dc938a34f0394c7d0acc9bee9ca]) successfully deleted
I20251023 02:08:27.181185 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:27.181751 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:27.181852 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:29.182338 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:29.182974 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:29.183087 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:31.183416 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:31.183925 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:31.184046 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:33.184410 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:33.185009 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:33.185119 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:35.185715 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:35.186307 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:35.186412 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:37.186731 17670 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:08:37.187245 17670 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:08:37.187350 17670 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:37.332906 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:08:37.339910 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:37.340167 16007 tablet_replica.cc:333] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b: stopping tablet replica
I20251023 02:08:37.340350 16007 raft_consensus.cc:2243] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:37.340620 16007 raft_consensus.cc:2272] T 580961fac15348b3a25d15dbd90a702a P c56bee725057458db50e4cd84930542b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.354642 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:08:37.357429 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:08:37.365692 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:37.365957 16007 tablet_replica.cc:333] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e: stopping tablet replica
I20251023 02:08:37.366107 16007 raft_consensus.cc:2243] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:37.366366 16007 raft_consensus.cc:2272] T dda04d421be546d5a713d9886d8abdfa P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.366849 16007 tablet_replica.cc:333] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e: stopping tablet replica
I20251023 02:08:37.366976 16007 raft_consensus.cc:2243] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:37.367107 16007 raft_consensus.cc:2272] T 580961fac15348b3a25d15dbd90a702a P 578d6b86d6234a79a4f97d604936681e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.380306 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:08:37.382987 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:08:37.390992 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:37.391187 16007 tablet_replica.cc:333] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66: stopping tablet replica
I20251023 02:08:37.391330 16007 raft_consensus.cc:2243] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:37.391569 16007 raft_consensus.cc:2272] T dda04d421be546d5a713d9886d8abdfa P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.391980 16007 tablet_replica.cc:333] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66: stopping tablet replica
I20251023 02:08:37.392164 16007 raft_consensus.cc:2243] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:37.392297 16007 raft_consensus.cc:2272] T 580961fac15348b3a25d15dbd90a702a P 39cd3cbd74d34c0092b35503bd1e1d66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.407634 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:08:37.417841 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:08:37.425698 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:08:37.425892 16007 tablet_replica.cc:333] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20: stopping tablet replica
I20251023 02:08:37.426021 16007 raft_consensus.cc:2243] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:08:37.426191 16007 raft_consensus.cc:2272] T dda04d421be546d5a713d9886d8abdfa P ff9a3db794944037963a1cc0d1bf4f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.438907 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:08:37.441252 16007 master.cc:561] Master@127.15.161.254:41203 shutting down...
I20251023 02:08:37.447695 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:37.447834 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:37.447914 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 605228fd4dc14d699d88c9825181abc4: stopping tablet replica
I20251023 02:08:37.460493 16007 master.cc:583] Master@127.15.161.254:41203 shutdown complete.
[ OK ] AutoRebalancerTest.MovesScheduledIfAddTserver (26351 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTservers
I20251023 02:08:37.470428 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:36861
I20251023 02:08:37.470829 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:37.472921 18062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:37.472923 18061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:37.473321 18064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:37.473558 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:37.473778 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:37.473821 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:37.473865 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185317473861 us; error 0 us; skew 500 ppm
I20251023 02:08:37.473996 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:37.475041 16007 webserver.cc:492] Webserver started at http://127.15.161.254:33887/ using document root <none> and password file <none>
I20251023 02:08:37.475240 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:37.475291 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:37.475402 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:37.475972 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "7c047e86f3cd466da0576f3913251dc2"
format_stamp: "Formatted at 2025-10-23 02:08:37 on dist-test-slave-bfbt"
I20251023 02:08:37.478097 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:08:37.479403 18069 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:37.479712 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:37.479799 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "7c047e86f3cd466da0576f3913251dc2"
format_stamp: "Formatted at 2025-10-23 02:08:37 on dist-test-slave-bfbt"
I20251023 02:08:37.479907 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:37.486737 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:37.487136 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:37.497888 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:36861
I20251023 02:08:37.497905 18120 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:36861 every 8 connection(s)
I20251023 02:08:37.499109 18121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:37.502449 18121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2: Bootstrap starting.
I20251023 02:08:37.503358 18121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:37.504712 18121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2: No bootstrap required, opened a new log
I20251023 02:08:37.505229 18121 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c047e86f3cd466da0576f3913251dc2" member_type: VOTER }
I20251023 02:08:37.505340 18121 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:37.505383 18121 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c047e86f3cd466da0576f3913251dc2, State: Initialized, Role: FOLLOWER
I20251023 02:08:37.505533 18121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c047e86f3cd466da0576f3913251dc2" member_type: VOTER }
I20251023 02:08:37.505615 18121 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:08:37.505671 18121 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:08:37.505723 18121 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:37.507165 18121 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c047e86f3cd466da0576f3913251dc2" member_type: VOTER }
I20251023 02:08:37.507315 18121 leader_election.cc:304] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c047e86f3cd466da0576f3913251dc2; no voters:
I20251023 02:08:37.507671 18121 leader_election.cc:290] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:08:37.507776 18124 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:37.508296 18124 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 1 LEADER]: Becoming Leader. State: Replica: 7c047e86f3cd466da0576f3913251dc2, State: Running, Role: LEADER
I20251023 02:08:37.508553 18124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c047e86f3cd466da0576f3913251dc2" member_type: VOTER }
I20251023 02:08:37.508646 18121 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:08:37.509781 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c047e86f3cd466da0576f3913251dc2. Latest consensus state: current_term: 1 leader_uuid: "7c047e86f3cd466da0576f3913251dc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c047e86f3cd466da0576f3913251dc2" member_type: VOTER } }
I20251023 02:08:37.509974 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:37.510573 18125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c047e86f3cd466da0576f3913251dc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c047e86f3cd466da0576f3913251dc2" member_type: VOTER } }
I20251023 02:08:37.510659 18130 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:08:37.510784 18125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:37.511749 18130 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:08:37.514451 18130 catalog_manager.cc:1357] Generated new cluster ID: b9a2ae4e97014ce0a901f8db82736780
I20251023 02:08:37.514525 18130 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:08:37.515358 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:08:37.520078 18130 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:08:37.520673 18130 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:08:37.524899 18130 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2: Generated new TSK 0
I20251023 02:08:37.525059 18130 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:08:37.531103 16007 internal_mini_cluster.cc:371] 0 TS(s) registered with all masters after 5.3523e-05s
I20251023 02:08:49.611958 16007 master.cc:561] Master@127.15.161.254:36861 shutting down...
I20251023 02:08:49.623340 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:08:49.623559 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:08:49.623641 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7c047e86f3cd466da0576f3913251dc2: stopping tablet replica
I20251023 02:08:49.636864 16007 master.cc:583] Master@127.15.161.254:36861 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTservers (12173 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTablets
I20251023 02:08:49.643850 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:33867
I20251023 02:08:49.644248 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:49.646523 18144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.646615 18146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.646531 18143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:49.648984 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:49.651041 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:49.651110 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:49.651151 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185329651137 us; error 0 us; skew 500 ppm
I20251023 02:08:49.651264 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:49.652196 16007 webserver.cc:492] Webserver started at http://127.15.161.254:44687/ using document root <none> and password file <none>
I20251023 02:08:49.652364 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:49.652420 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:49.652513 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:49.653045 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "e623e301ad4a46d78bdf49efd3bac560"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.655104 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:08:49.656322 18151 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.656608 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:49.656698 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "e623e301ad4a46d78bdf49efd3bac560"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.656801 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:49.671196 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:49.671614 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:49.680989 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:33867
I20251023 02:08:49.681005 18202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:33867 every 8 connection(s)
I20251023 02:08:49.682081 18203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:08:49.685443 18203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560: Bootstrap starting.
I20251023 02:08:49.686406 18203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560: Neither blocks nor log segments found. Creating new log.
I20251023 02:08:49.687512 18203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560: No bootstrap required, opened a new log
I20251023 02:08:49.687981 18203 raft_consensus.cc:359] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e623e301ad4a46d78bdf49efd3bac560" member_type: VOTER }
I20251023 02:08:49.688092 18203 raft_consensus.cc:385] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:08:49.688158 18203 raft_consensus.cc:740] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e623e301ad4a46d78bdf49efd3bac560, State: Initialized, Role: FOLLOWER
I20251023 02:08:49.688324 18203 consensus_queue.cc:260] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e623e301ad4a46d78bdf49efd3bac560" member_type: VOTER }
I20251023 02:08:49.688427 18203 raft_consensus.cc:399] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:08:49.688495 18203 raft_consensus.cc:493] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:08:49.688576 18203 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:08:49.689975 18203 raft_consensus.cc:515] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e623e301ad4a46d78bdf49efd3bac560" member_type: VOTER }
I20251023 02:08:49.690160 18203 leader_election.cc:304] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e623e301ad4a46d78bdf49efd3bac560; no voters:
I20251023 02:08:49.690511 18203 leader_election.cc:290] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:08:49.690593 18206 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:08:49.690987 18206 raft_consensus.cc:697] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 1 LEADER]: Becoming Leader. State: Replica: e623e301ad4a46d78bdf49efd3bac560, State: Running, Role: LEADER
I20251023 02:08:49.691215 18206 consensus_queue.cc:237] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e623e301ad4a46d78bdf49efd3bac560" member_type: VOTER }
I20251023 02:08:49.691283 18203 sys_catalog.cc:565] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:08:49.691823 18207 sys_catalog.cc:455] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e623e301ad4a46d78bdf49efd3bac560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e623e301ad4a46d78bdf49efd3bac560" member_type: VOTER } }
I20251023 02:08:49.691864 18208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e623e301ad4a46d78bdf49efd3bac560. Latest consensus state: current_term: 1 leader_uuid: "e623e301ad4a46d78bdf49efd3bac560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e623e301ad4a46d78bdf49efd3bac560" member_type: VOTER } }
I20251023 02:08:49.692060 18207 sys_catalog.cc:458] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:49.692090 18208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [sys.catalog]: This master's current role is: LEADER
I20251023 02:08:49.692991 18211 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:08:49.694381 18211 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:08:49.696106 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:08:49.697299 18211 catalog_manager.cc:1357] Generated new cluster ID: afb8157597c24c6595b4db5589df2062
I20251023 02:08:49.697378 18211 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:08:49.703750 18211 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:08:49.704352 18211 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:08:49.715045 18211 catalog_manager.cc:6022] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560: Generated new TSK 0
I20251023 02:08:49.715235 18211 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:08:49.728705 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:49.731004 18224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.731362 18225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.731689 18227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:49.732204 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:49.732381 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:49.732436 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:49.732489 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185329732467 us; error 0 us; skew 500 ppm
I20251023 02:08:49.732625 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:49.733623 16007 webserver.cc:492] Webserver started at http://127.15.161.193:37255/ using document root <none> and password file <none>
I20251023 02:08:49.733829 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:49.733897 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:49.734017 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:49.734606 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "4fc1bacdcf9744cdab773fbb6c0b0836"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.736641 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20251023 02:08:49.737854 18232 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.738142 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:49.738243 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "4fc1bacdcf9744cdab773fbb6c0b0836"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.738336 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:49.751106 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:49.751595 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:49.752029 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:49.760515 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:49.760568 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.760637 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:49.760679 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.770041 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:44855
I20251023 02:08:49.770063 18294 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:44855 every 8 connection(s)
I20251023 02:08:49.771536 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:49.773626 18299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.773800 18300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:49.774854 16007 server_base.cc:1047] running on GCE node
W20251023 02:08:49.774891 18302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:49.775228 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:49.775280 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:49.775334 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185329775330 us; error 0 us; skew 500 ppm
I20251023 02:08:49.775460 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:49.776547 16007 webserver.cc:492] Webserver started at http://127.15.161.194:45067/ using document root <none> and password file <none>
I20251023 02:08:49.776741 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:49.776790 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:49.776890 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:49.776983 18295 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33867
I20251023 02:08:49.777081 18295 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:49.777364 18295 heartbeater.cc:507] Master 127.15.161.254:33867 requested a full tablet report, sending...
I20251023 02:08:49.777499 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "7234614f87a5483780c0405c780cd0fc"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.778198 18168 ts_manager.cc:194] Registered new tserver with Master: 4fc1bacdcf9744cdab773fbb6c0b0836 (127.15.161.193:44855)
I20251023 02:08:49.779139 18168 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49464
I20251023 02:08:49.779981 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:08:49.781334 18307 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.781628 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:49.781698 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "7234614f87a5483780c0405c780cd0fc"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.781773 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:49.786891 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:49.787281 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:49.787695 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:49.788290 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:49.788336 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.788393 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:49.788424 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.799872 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:37847
I20251023 02:08:49.799902 18369 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:37847 every 8 connection(s)
I20251023 02:08:49.801151 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:08:49.803062 18373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.803508 18374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:08:49.803630 18376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:08:49.806775 16007 server_base.cc:1047] running on GCE node
I20251023 02:08:49.806912 18370 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33867
I20251023 02:08:49.807031 18370 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:49.807071 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:08:49.807139 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:08:49.807183 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185329807176 us; error 0 us; skew 500 ppm
I20251023 02:08:49.807308 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:08:49.807394 18370 heartbeater.cc:507] Master 127.15.161.254:33867 requested a full tablet report, sending...
I20251023 02:08:49.808115 18168 ts_manager.cc:194] Registered new tserver with Master: 7234614f87a5483780c0405c780cd0fc (127.15.161.194:37847)
I20251023 02:08:49.808449 16007 webserver.cc:492] Webserver started at http://127.15.161.195:35757/ using document root <none> and password file <none>
I20251023 02:08:49.808688 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:08:49.808754 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:08:49.808787 18168 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49466
I20251023 02:08:49.808852 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:08:49.809280 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "ec34e2839f394597942ce2cfe3f045f1"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.811347 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:08:49.812480 18381 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.812768 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:08:49.812870 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "ec34e2839f394597942ce2cfe3f045f1"
format_stamp: "Formatted at 2025-10-23 02:08:49 on dist-test-slave-bfbt"
I20251023 02:08:49.812949 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:08:49.821822 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:08:49.822207 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:08:49.822583 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:08:49.823170 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:08:49.823213 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.823269 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:08:49.823303 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:08:49.833869 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:44603
I20251023 02:08:49.833889 18443 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:44603 every 8 connection(s)
I20251023 02:08:49.839002 18444 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33867
I20251023 02:08:49.839099 18444 heartbeater.cc:461] Registering TS with master...
I20251023 02:08:49.839367 18444 heartbeater.cc:507] Master 127.15.161.254:33867 requested a full tablet report, sending...
I20251023 02:08:49.840075 18168 ts_manager.cc:194] Registered new tserver with Master: ec34e2839f394597942ce2cfe3f045f1 (127.15.161.195:44603)
I20251023 02:08:49.840108 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005525355s
I20251023 02:08:49.840770 18168 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49482
I20251023 02:08:50.780529 18295 heartbeater.cc:499] Master 127.15.161.254:33867 was elected leader, sending a full tablet report...
I20251023 02:08:50.809727 18370 heartbeater.cc:499] Master 127.15.161.254:33867 was elected leader, sending a full tablet report...
I20251023 02:08:50.841925 18444 heartbeater.cc:499] Master 127.15.161.254:33867 was elected leader, sending a full tablet report...
I20251023 02:08:51.696002 18221 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:53.696411 18221 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:55.696892 18221 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:57.697332 18221 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:08:59.697824 18221 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:01.698355 18221 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:01.921074 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:09:01.932188 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:01.945524 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:09:01.947752 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:09:01.954484 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:01.966552 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:09:01.968783 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:09:01.976130 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:01.988042 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:09:01.990058 16007 master.cc:561] Master@127.15.161.254:33867 shutting down...
I20251023 02:09:01.996884 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:09:01.997049 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:01.997125 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P e623e301ad4a46d78bdf49efd3bac560: stopping tablet replica
I20251023 02:09:02.010164 16007 master.cc:583] Master@127.15.161.254:33867 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTablets (12373 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne
I20251023 02:09:02.017781 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:45881
I20251023 02:09:02.018178 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:02.020073 18448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.020126 18449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.020680 18451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.021250 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:02.021487 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:02.021549 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:02.021600 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185342021585 us; error 0 us; skew 500 ppm
I20251023 02:09:02.021716 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:02.022872 16007 webserver.cc:492] Webserver started at http://127.15.161.254:40529/ using document root <none> and password file <none>
I20251023 02:09:02.023046 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:02.023098 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:02.023182 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:02.023655 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.025604 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:09:02.026974 18456 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.027276 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:02.027396 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.027493 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:02.039639 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:02.040037 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:02.050103 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:45881
I20251023 02:09:02.050151 18507 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:45881 every 8 connection(s)
I20251023 02:09:02.051081 18508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:02.054437 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef: Bootstrap starting.
I20251023 02:09:02.055490 18508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:02.056731 18508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef: No bootstrap required, opened a new log
I20251023 02:09:02.057246 18508 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" member_type: VOTER }
I20251023 02:09:02.057351 18508 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:02.057422 18508 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5ee329ec9843aa8f10ebe0a4e4fcef, State: Initialized, Role: FOLLOWER
I20251023 02:09:02.057608 18508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" member_type: VOTER }
I20251023 02:09:02.057719 18508 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:09:02.057790 18508 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:09:02.057879 18508 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:02.059314 18508 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" member_type: VOTER }
I20251023 02:09:02.059477 18508 leader_election.cc:304] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b5ee329ec9843aa8f10ebe0a4e4fcef; no voters:
I20251023 02:09:02.059774 18508 leader_election.cc:290] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:09:02.059865 18511 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:02.060297 18511 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 1 LEADER]: Becoming Leader. State: Replica: 1b5ee329ec9843aa8f10ebe0a4e4fcef, State: Running, Role: LEADER
I20251023 02:09:02.060482 18508 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:09:02.060485 18511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" member_type: VOTER }
I20251023 02:09:02.061228 18513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b5ee329ec9843aa8f10ebe0a4e4fcef. Latest consensus state: current_term: 1 leader_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" member_type: VOTER } }
I20251023 02:09:02.061269 18512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b5ee329ec9843aa8f10ebe0a4e4fcef" member_type: VOTER } }
I20251023 02:09:02.061510 18513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [sys.catalog]: This master's current role is: LEADER
I20251023 02:09:02.062055 18512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [sys.catalog]: This master's current role is: LEADER
I20251023 02:09:02.065188 18516 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:09:02.065289 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:09:02.066361 18516 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:09:02.068754 18516 catalog_manager.cc:1357] Generated new cluster ID: 88d49c4170a54acbb141982192064498
I20251023 02:09:02.068820 18516 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:09:02.102083 18516 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:09:02.102814 18516 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:09:02.124182 18516 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef: Generated new TSK 0
I20251023 02:09:02.124361 18516 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:09:02.129740 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:02.132027 18529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.132230 18530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.132735 18532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.133616 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:02.142565 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:02.142620 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:02.142664 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185342142650 us; error 0 us; skew 500 ppm
I20251023 02:09:02.142769 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:02.143723 16007 webserver.cc:492] Webserver started at http://127.15.161.193:46179/ using document root <none> and password file <none>
I20251023 02:09:02.143888 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:02.143931 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:02.144007 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:02.144449 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "b4c9a79b94dd46d18cc20e658dbcb00f"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.146618 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:09:02.147878 18537 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.148147 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:02.148217 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "b4c9a79b94dd46d18cc20e658dbcb00f"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.148303 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:02.166469 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:02.166836 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:02.167230 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:02.167915 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:02.167960 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.168012 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:02.168053 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.177177 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:41845
I20251023 02:09:02.177198 18599 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:41845 every 8 connection(s)
I20251023 02:09:02.178413 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:02.180457 18605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.180433 18604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.181090 18607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.181942 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:02.182235 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:02.182289 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:02.182334 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185342182330 us; error 0 us; skew 500 ppm
I20251023 02:09:02.182468 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:02.183483 16007 webserver.cc:492] Webserver started at http://127.15.161.194:44899/ using document root <none> and password file <none>
I20251023 02:09:02.183641 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:02.183694 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:02.183790 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:02.184229 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "5e699b420b35457fbdcd39dc6bc99b98"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.186089 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 02:09:02.187188 18600 heartbeater.cc:344] Connected to a master server at 127.15.161.254:45881
I20251023 02:09:02.187288 18600 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:02.187618 18612 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.187644 18600 heartbeater.cc:507] Master 127.15.161.254:45881 requested a full tablet report, sending...
I20251023 02:09:02.187916 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:09:02.188006 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "5e699b420b35457fbdcd39dc6bc99b98"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.188090 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:02.188349 18473 ts_manager.cc:194] Registered new tserver with Master: b4c9a79b94dd46d18cc20e658dbcb00f (127.15.161.193:41845)
I20251023 02:09:02.189366 18473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56500
I20251023 02:09:02.198963 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:02.199301 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:02.199679 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:02.200307 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:02.200356 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.200413 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:02.200448 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.210731 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:39267
I20251023 02:09:02.210763 18674 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:39267 every 8 connection(s)
I20251023 02:09:02.212071 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:02.214344 18678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.215108 18679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.216286 16007 server_base.cc:1047] running on GCE node
W20251023 02:09:02.216416 18681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.216742 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:02.216794 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:02.216822 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185342216819 us; error 0 us; skew 500 ppm
I20251023 02:09:02.216960 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:02.218158 16007 webserver.cc:492] Webserver started at http://127.15.161.195:43641/ using document root <none> and password file <none>
I20251023 02:09:02.218359 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:02.218428 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:02.218539 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:02.218636 18675 heartbeater.cc:344] Connected to a master server at 127.15.161.254:45881
I20251023 02:09:02.218743 18675 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:02.219055 18675 heartbeater.cc:507] Master 127.15.161.254:45881 requested a full tablet report, sending...
I20251023 02:09:02.219197 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "b54767b26cdf4b0a8653b34c290d5b4e"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.219846 18473 ts_manager.cc:194] Registered new tserver with Master: 5e699b420b35457fbdcd39dc6bc99b98 (127.15.161.194:39267)
I20251023 02:09:02.220757 18473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56514
I20251023 02:09:02.221814 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:09:02.223205 18686 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.223475 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:02.223551 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "b54767b26cdf4b0a8653b34c290d5b4e"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.223639 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:02.247505 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:02.247879 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:02.248284 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:02.248968 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:02.249013 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.249094 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:02.249135 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.259194 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:45241
I20251023 02:09:02.259229 18748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:45241 every 8 connection(s)
I20251023 02:09:02.260452 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:02.262861 18752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.263880 16007 server_base.cc:1047] running on GCE node
W20251023 02:09:02.263880 18753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:02.264446 18756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:02.264760 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:02.264816 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:02.264863 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185342264847 us; error 0 us; skew 500 ppm
I20251023 02:09:02.264990 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:02.265928 18749 heartbeater.cc:344] Connected to a master server at 127.15.161.254:45881
I20251023 02:09:02.266042 18749 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:02.266214 16007 webserver.cc:492] Webserver started at http://127.15.161.196:43339/ using document root <none> and password file <none>
I20251023 02:09:02.266386 18749 heartbeater.cc:507] Master 127.15.161.254:45881 requested a full tablet report, sending...
I20251023 02:09:02.266428 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:02.266475 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:02.266572 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:02.267097 18473 ts_manager.cc:194] Registered new tserver with Master: b54767b26cdf4b0a8653b34c290d5b4e (127.15.161.195:45241)
I20251023 02:09:02.267128 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "0d77db9caafa46a7a9a0ed409ede2445"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.267699 18473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56522
I20251023 02:09:02.269176 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 02:09:02.270474 18760 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.270766 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:02.270876 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "0d77db9caafa46a7a9a0ed409ede2445"
format_stamp: "Formatted at 2025-10-23 02:09:02 on dist-test-slave-bfbt"
I20251023 02:09:02.270978 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:02.284754 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:02.285109 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:02.285511 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:02.286092 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:02.286161 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.286228 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:02.286267 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:02.296347 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:36511
I20251023 02:09:02.296376 18822 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:36511 every 8 connection(s)
I20251023 02:09:02.301241 18823 heartbeater.cc:344] Connected to a master server at 127.15.161.254:45881
I20251023 02:09:02.301317 18823 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:02.301554 18823 heartbeater.cc:507] Master 127.15.161.254:45881 requested a full tablet report, sending...
I20251023 02:09:02.302237 18473 ts_manager.cc:194] Registered new tserver with Master: 0d77db9caafa46a7a9a0ed409ede2445 (127.15.161.196:36511)
I20251023 02:09:02.302583 16007 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.00551987s
I20251023 02:09:02.302877 18473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56528
I20251023 02:09:03.190416 18600 heartbeater.cc:499] Master 127.15.161.254:45881 was elected leader, sending a full tablet report...
I20251023 02:09:03.221956 18675 heartbeater.cc:499] Master 127.15.161.254:45881 was elected leader, sending a full tablet report...
I20251023 02:09:03.268630 18749 heartbeater.cc:499] Master 127.15.161.254:45881 was elected leader, sending a full tablet report...
I20251023 02:09:03.303943 18823 heartbeater.cc:499] Master 127.15.161.254:45881 was elected leader, sending a full tablet report...
I20251023 02:09:03.335605 16007 test_util.cc:276] Using random seed: -356127566
I20251023 02:09:03.346215 18473 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33866:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251023 02:09:03.369318 18565 tablet_service.cc:1505] Processing CreateTablet for tablet efe57b9a03e24a939a8fc1fb3bc385ed (DEFAULT_TABLE table=test-workload [id=9ffd780d7eb14047a0d7dd093ca2db64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:09:03.369740 18565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe57b9a03e24a939a8fc1fb3bc385ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:03.374756 18640 tablet_service.cc:1505] Processing CreateTablet for tablet efe57b9a03e24a939a8fc1fb3bc385ed (DEFAULT_TABLE table=test-workload [id=9ffd780d7eb14047a0d7dd093ca2db64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:09:03.375018 18640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe57b9a03e24a939a8fc1fb3bc385ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:03.375655 18714 tablet_service.cc:1505] Processing CreateTablet for tablet efe57b9a03e24a939a8fc1fb3bc385ed (DEFAULT_TABLE table=test-workload [id=9ffd780d7eb14047a0d7dd093ca2db64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:09:03.375989 18714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efe57b9a03e24a939a8fc1fb3bc385ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:03.378500 18844 tablet_bootstrap.cc:492] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: Bootstrap starting.
I20251023 02:09:03.380033 18844 tablet_bootstrap.cc:654] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:03.381140 18845 tablet_bootstrap.cc:492] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98: Bootstrap starting.
I20251023 02:09:03.381541 18844 tablet_bootstrap.cc:492] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: No bootstrap required, opened a new log
I20251023 02:09:03.381683 18844 ts_tablet_manager.cc:1403] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:09:03.382480 18844 raft_consensus.cc:359] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.382650 18844 raft_consensus.cc:385] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:03.382725 18844 raft_consensus.cc:740] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c9a79b94dd46d18cc20e658dbcb00f, State: Initialized, Role: FOLLOWER
I20251023 02:09:03.382910 18845 tablet_bootstrap.cc:654] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:03.382951 18844 consensus_queue.cc:260] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.383714 18844 ts_tablet_manager.cc:1434] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:09:03.384270 18847 tablet_bootstrap.cc:492] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e: Bootstrap starting.
I20251023 02:09:03.386343 18847 tablet_bootstrap.cc:654] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:03.386446 18845 tablet_bootstrap.cc:492] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98: No bootstrap required, opened a new log
I20251023 02:09:03.386567 18845 ts_tablet_manager.cc:1403] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251023 02:09:03.387367 18845 raft_consensus.cc:359] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.387501 18845 raft_consensus.cc:385] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:03.387563 18845 raft_consensus.cc:740] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e699b420b35457fbdcd39dc6bc99b98, State: Initialized, Role: FOLLOWER
I20251023 02:09:03.387753 18845 consensus_queue.cc:260] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.387869 18847 tablet_bootstrap.cc:492] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e: No bootstrap required, opened a new log
I20251023 02:09:03.387953 18847 ts_tablet_manager.cc:1403] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:09:03.388363 18845 ts_tablet_manager.cc:1434] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:03.388588 18847 raft_consensus.cc:359] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.388713 18847 raft_consensus.cc:385] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:03.388760 18847 raft_consensus.cc:740] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54767b26cdf4b0a8653b34c290d5b4e, State: Initialized, Role: FOLLOWER
I20251023 02:09:03.388926 18847 consensus_queue.cc:260] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.389525 18847 ts_tablet_manager.cc:1434] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20251023 02:09:03.428908 18601 tablet.cc:2378] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 02:09:03.537308 18852 raft_consensus.cc:493] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:03.537424 18852 raft_consensus.cc:515] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.537992 18852 leader_election.cc:290] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4c9a79b94dd46d18cc20e658dbcb00f (127.15.161.193:41845), 5e699b420b35457fbdcd39dc6bc99b98 (127.15.161.194:39267)
I20251023 02:09:03.543969 18575 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe57b9a03e24a939a8fc1fb3bc385ed" candidate_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" is_pre_election: true
I20251023 02:09:03.544209 18575 raft_consensus.cc:2468] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54767b26cdf4b0a8653b34c290d5b4e in term 0.
I20251023 02:09:03.544309 18650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe57b9a03e24a939a8fc1fb3bc385ed" candidate_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e699b420b35457fbdcd39dc6bc99b98" is_pre_election: true
I20251023 02:09:03.544535 18650 raft_consensus.cc:2468] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b54767b26cdf4b0a8653b34c290d5b4e in term 0.
I20251023 02:09:03.544620 18690 leader_election.cc:304] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4c9a79b94dd46d18cc20e658dbcb00f, b54767b26cdf4b0a8653b34c290d5b4e; no voters:
I20251023 02:09:03.544868 18852 raft_consensus.cc:2804] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:03.544930 18852 raft_consensus.cc:493] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:03.544997 18852 raft_consensus.cc:3060] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:03.546571 18852 raft_consensus.cc:515] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.546996 18852 leader_election.cc:290] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [CANDIDATE]: Term 1 election: Requested vote from peers b4c9a79b94dd46d18cc20e658dbcb00f (127.15.161.193:41845), 5e699b420b35457fbdcd39dc6bc99b98 (127.15.161.194:39267)
I20251023 02:09:03.547396 18575 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe57b9a03e24a939a8fc1fb3bc385ed" candidate_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f"
I20251023 02:09:03.547396 18650 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "efe57b9a03e24a939a8fc1fb3bc385ed" candidate_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e699b420b35457fbdcd39dc6bc99b98"
I20251023 02:09:03.547562 18650 raft_consensus.cc:3060] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:03.547569 18575 raft_consensus.cc:3060] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:03.549870 18575 raft_consensus.cc:2468] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54767b26cdf4b0a8653b34c290d5b4e in term 1.
I20251023 02:09:03.549886 18650 raft_consensus.cc:2468] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b54767b26cdf4b0a8653b34c290d5b4e in term 1.
I20251023 02:09:03.550211 18688 leader_election.cc:304] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5e699b420b35457fbdcd39dc6bc99b98, b4c9a79b94dd46d18cc20e658dbcb00f, b54767b26cdf4b0a8653b34c290d5b4e; no voters:
I20251023 02:09:03.550393 18852 raft_consensus.cc:2804] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:03.550685 18852 raft_consensus.cc:697] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 1 LEADER]: Becoming Leader. State: Replica: b54767b26cdf4b0a8653b34c290d5b4e, State: Running, Role: LEADER
I20251023 02:09:03.550868 18852 consensus_queue.cc:237] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } }
I20251023 02:09:03.552979 18472 catalog_manager.cc:5649] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e reported cstate change: term changed from 0 to 1, leader changed from <none> to b54767b26cdf4b0a8653b34c290d5b4e (127.15.161.195). New cstate: current_term: 1 leader_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54767b26cdf4b0a8653b34c290d5b4e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:03.928123 18852 consensus_queue.cc:1048] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c9a79b94dd46d18cc20e658dbcb00f" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 41845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:03.932581 18855 consensus_queue.cc:1048] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e699b420b35457fbdcd39dc6bc99b98" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:04.064880 18526 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:04.065258 18526 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:04.065378 18526 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:06.065672 18526 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:06.066046 18526 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:06.066152 18526 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:08.066540 18526 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:08.066906 18526 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:08.066990 18526 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:10.067343 18526 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:10.067713 18526 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:10.067814 18526 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:12.068172 18526 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:12.068565 18526 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:12.068650 18526 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:14.069069 18526 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:14.069479 18526 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:14.069567 18526 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:14.641208 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:09:14.651938 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:14.652149 16007 tablet_replica.cc:333] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f: stopping tablet replica
I20251023 02:09:14.652305 16007 raft_consensus.cc:2243] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:14.652465 16007 raft_consensus.cc:2272] T efe57b9a03e24a939a8fc1fb3bc385ed P b4c9a79b94dd46d18cc20e658dbcb00f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:14.665555 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:09:14.667887 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:09:14.675592 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:14.675767 16007 tablet_replica.cc:333] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98: stopping tablet replica
I20251023 02:09:14.675917 16007 raft_consensus.cc:2243] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:14.676064 16007 raft_consensus.cc:2272] T efe57b9a03e24a939a8fc1fb3bc385ed P 5e699b420b35457fbdcd39dc6bc99b98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:14.689808 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:09:14.691987 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:09:14.708563 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:14.708741 16007 tablet_replica.cc:333] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e: stopping tablet replica
I20251023 02:09:14.708889 16007 raft_consensus.cc:2243] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:09:14.709131 16007 raft_consensus.cc:2272] T efe57b9a03e24a939a8fc1fb3bc385ed P b54767b26cdf4b0a8653b34c290d5b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:14.721978 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:09:14.724392 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:09:14.732591 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:14.744882 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:09:14.747112 16007 master.cc:561] Master@127.15.161.254:45881 shutting down...
I20251023 02:09:14.753582 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:09:14.753741 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:14.753836 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1b5ee329ec9843aa8f10ebe0a4e4fcef: stopping tablet replica
I20251023 02:09:14.766561 16007 master.cc:583] Master@127.15.161.254:45881 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne (12757 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy
I20251023 02:09:14.775745 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:38515
I20251023 02:09:14.776104 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:14.778059 18866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.778260 18867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.778257 18869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:14.778348 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:14.778761 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:14.778812 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:14.778849 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185354778846 us; error 0 us; skew 500 ppm
I20251023 02:09:14.778973 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:14.779891 16007 webserver.cc:492] Webserver started at http://127.15.161.254:35095/ using document root <none> and password file <none>
I20251023 02:09:14.780069 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:14.780117 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:14.780221 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:14.780783 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "c0a3341b82b94b06b3346a8a0b95e1ae"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.782856 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:09:14.784267 18874 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.784564 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:14.784634 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "c0a3341b82b94b06b3346a8a0b95e1ae"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.784721 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:14.794621 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:14.794974 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:14.803581 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:38515
I20251023 02:09:14.803602 18925 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:38515 every 8 connection(s)
I20251023 02:09:14.804581 18926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:14.807699 18926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae: Bootstrap starting.
I20251023 02:09:14.808588 18926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:14.809656 18926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae: No bootstrap required, opened a new log
I20251023 02:09:14.810071 18926 raft_consensus.cc:359] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" member_type: VOTER }
I20251023 02:09:14.810209 18926 raft_consensus.cc:385] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:14.810249 18926 raft_consensus.cc:740] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a3341b82b94b06b3346a8a0b95e1ae, State: Initialized, Role: FOLLOWER
I20251023 02:09:14.810365 18926 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" member_type: VOTER }
I20251023 02:09:14.810441 18926 raft_consensus.cc:399] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:09:14.810474 18926 raft_consensus.cc:493] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:09:14.810515 18926 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:14.811833 18926 raft_consensus.cc:515] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" member_type: VOTER }
I20251023 02:09:14.811964 18926 leader_election.cc:304] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0a3341b82b94b06b3346a8a0b95e1ae; no voters:
I20251023 02:09:14.812211 18926 leader_election.cc:290] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:09:14.812310 18929 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:14.812696 18929 raft_consensus.cc:697] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 1 LEADER]: Becoming Leader. State: Replica: c0a3341b82b94b06b3346a8a0b95e1ae, State: Running, Role: LEADER
I20251023 02:09:14.812896 18926 sys_catalog.cc:565] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:09:14.812928 18929 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" member_type: VOTER }
I20251023 02:09:14.813655 18931 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0a3341b82b94b06b3346a8a0b95e1ae. Latest consensus state: current_term: 1 leader_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" member_type: VOTER } }
I20251023 02:09:14.813738 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3341b82b94b06b3346a8a0b95e1ae" member_type: VOTER } }
I20251023 02:09:14.813863 18931 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [sys.catalog]: This master's current role is: LEADER
I20251023 02:09:14.813989 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [sys.catalog]: This master's current role is: LEADER
I20251023 02:09:14.815090 18935 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:09:14.816486 18935 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:09:14.817417 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:09:14.819044 18935 catalog_manager.cc:1357] Generated new cluster ID: 267b16935a714fec92dfd34ee1230063
I20251023 02:09:14.819110 18935 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:09:14.846061 18935 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:09:14.846704 18935 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:09:14.859340 18935 catalog_manager.cc:6022] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae: Generated new TSK 0
I20251023 02:09:14.859498 18935 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:09:14.882128 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:14.884299 18947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.884548 18948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.884846 18950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:14.885560 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:14.885751 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:14.885807 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:14.885851 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185354885847 us; error 0 us; skew 500 ppm
I20251023 02:09:14.885977 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:14.886929 16007 webserver.cc:492] Webserver started at http://127.15.161.193:34691/ using document root <none> and password file <none>
I20251023 02:09:14.887099 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:14.887148 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:14.887251 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:14.887756 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "71ccde811b2d47d3b8296744839f3511"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.889986 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:09:14.891331 18955 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.891672 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:14.891747 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "71ccde811b2d47d3b8296744839f3511"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.891821 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:14.899763 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:14.900105 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:14.900450 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:14.901078 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:14.901127 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.901170 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:14.901197 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.910871 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:45227
I20251023 02:09:14.910890 19017 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:45227 every 8 connection(s)
I20251023 02:09:14.912103 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:14.914346 19022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.914706 19023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.916739 19025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:14.916757 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:14.917142 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:14.917217 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:14.917261 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185354917257 us; error 0 us; skew 500 ppm
I20251023 02:09:14.917425 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:14.918536 16007 webserver.cc:492] Webserver started at http://127.15.161.194:37289/ using document root <none> and password file <none>
I20251023 02:09:14.918713 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:14.918767 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:14.918872 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:14.918905 19018 heartbeater.cc:344] Connected to a master server at 127.15.161.254:38515
I20251023 02:09:14.919013 19018 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:14.919281 19018 heartbeater.cc:507] Master 127.15.161.254:38515 requested a full tablet report, sending...
I20251023 02:09:14.919407 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "a55b84bcd37a44538cbdc7698bb4e9f6"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.919977 18891 ts_manager.cc:194] Registered new tserver with Master: 71ccde811b2d47d3b8296744839f3511 (127.15.161.193:45227)
I20251023 02:09:14.920874 18891 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44178
I20251023 02:09:14.921563 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:09:14.922959 19030 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.923226 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:09:14.923316 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "a55b84bcd37a44538cbdc7698bb4e9f6"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.923400 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:14.938488 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:14.938807 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:14.939173 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:14.939742 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:14.939788 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.939846 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:14.939877 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.948808 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:34961
I20251023 02:09:14.948850 19092 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:34961 every 8 connection(s)
I20251023 02:09:14.949942 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:14.951941 19096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.952378 19097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:14.953639 19099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:14.953806 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:14.954319 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:14.954375 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:14.954394 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185354954392 us; error 0 us; skew 500 ppm
I20251023 02:09:14.954495 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:14.955374 19093 heartbeater.cc:344] Connected to a master server at 127.15.161.254:38515
I20251023 02:09:14.955493 19093 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:14.955590 16007 webserver.cc:492] Webserver started at http://127.15.161.195:36431/ using document root <none> and password file <none>
I20251023 02:09:14.955762 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:14.955777 19093 heartbeater.cc:507] Master 127.15.161.254:38515 requested a full tablet report, sending...
I20251023 02:09:14.955825 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:14.955924 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:14.956457 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "69357d9a54b3453281b4f959b241f47e"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.956528 18891 ts_manager.cc:194] Registered new tserver with Master: a55b84bcd37a44538cbdc7698bb4e9f6 (127.15.161.194:34961)
I20251023 02:09:14.957398 18891 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44192
I20251023 02:09:14.958935 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:09:14.960124 19104 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.960407 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:14.960474 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "69357d9a54b3453281b4f959b241f47e"
format_stamp: "Formatted at 2025-10-23 02:09:14 on dist-test-slave-bfbt"
I20251023 02:09:14.960573 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:14.971123 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:14.971432 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:14.971825 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:14.972426 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:14.972481 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.972556 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:14.972611 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:14.981813 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:41191
I20251023 02:09:14.981833 19166 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:41191 every 8 connection(s)
I20251023 02:09:14.986937 19167 heartbeater.cc:344] Connected to a master server at 127.15.161.254:38515
I20251023 02:09:14.987040 19167 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:14.987298 19167 heartbeater.cc:507] Master 127.15.161.254:38515 requested a full tablet report, sending...
I20251023 02:09:14.987941 18891 ts_manager.cc:194] Registered new tserver with Master: 69357d9a54b3453281b4f959b241f47e (127.15.161.195:41191)
I20251023 02:09:14.988063 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005598845s
I20251023 02:09:14.988600 18891 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44206
I20251023 02:09:15.922142 19018 heartbeater.cc:499] Master 127.15.161.254:38515 was elected leader, sending a full tablet report...
I20251023 02:09:15.958597 19093 heartbeater.cc:499] Master 127.15.161.254:38515 was elected leader, sending a full tablet report...
I20251023 02:09:15.989593 19167 heartbeater.cc:499] Master 127.15.161.254:38515 was elected leader, sending a full tablet report...
I20251023 02:09:16.019972 16007 test_util.cc:276] Using random seed: -343443200
I20251023 02:09:16.028501 18891 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44232:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:09:16.029002 18891 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:09:16.057868 19132 tablet_service.cc:1505] Processing CreateTablet for tablet 18f11726025f423da0c7cf5b5d72c229 (DEFAULT_TABLE table=test-workload [id=3798a8864d1e4bdda60e41bd2d3919df]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:09:16.058229 19132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f11726025f423da0c7cf5b5d72c229. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:16.067613 18983 tablet_service.cc:1505] Processing CreateTablet for tablet 18f11726025f423da0c7cf5b5d72c229 (DEFAULT_TABLE table=test-workload [id=3798a8864d1e4bdda60e41bd2d3919df]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:09:16.067961 18983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f11726025f423da0c7cf5b5d72c229. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:16.070143 19188 tablet_bootstrap.cc:492] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e: Bootstrap starting.
I20251023 02:09:16.071419 19188 tablet_bootstrap.cc:654] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:16.072395 19058 tablet_service.cc:1505] Processing CreateTablet for tablet 18f11726025f423da0c7cf5b5d72c229 (DEFAULT_TABLE table=test-workload [id=3798a8864d1e4bdda60e41bd2d3919df]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:09:16.072710 19058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f11726025f423da0c7cf5b5d72c229. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:16.072721 19188 tablet_bootstrap.cc:492] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e: No bootstrap required, opened a new log
I20251023 02:09:16.072870 19188 ts_tablet_manager.cc:1403] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:09:16.073367 19188 raft_consensus.cc:359] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.073493 19188 raft_consensus.cc:385] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:16.073581 19188 raft_consensus.cc:740] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69357d9a54b3453281b4f959b241f47e, State: Initialized, Role: FOLLOWER
I20251023 02:09:16.073782 19188 consensus_queue.cc:260] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.082558 19192 tablet_bootstrap.cc:492] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6: Bootstrap starting.
I20251023 02:09:16.083231 19188 ts_tablet_manager.cc:1434] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20251023 02:09:16.083444 19190 tablet_bootstrap.cc:492] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511: Bootstrap starting.
I20251023 02:09:16.084151 19192 tablet_bootstrap.cc:654] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:16.084630 19190 tablet_bootstrap.cc:654] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:16.085624 19192 tablet_bootstrap.cc:492] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6: No bootstrap required, opened a new log
I20251023 02:09:16.085767 19192 ts_tablet_manager.cc:1403] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251023 02:09:16.086387 19192 raft_consensus.cc:359] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.086588 19192 raft_consensus.cc:385] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:16.086655 19192 raft_consensus.cc:740] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55b84bcd37a44538cbdc7698bb4e9f6, State: Initialized, Role: FOLLOWER
I20251023 02:09:16.086844 19192 consensus_queue.cc:260] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.087124 19190 tablet_bootstrap.cc:492] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511: No bootstrap required, opened a new log
I20251023 02:09:16.087229 19190 ts_tablet_manager.cc:1403] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:09:16.087422 19192 ts_tablet_manager.cc:1434] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:16.087716 19190 raft_consensus.cc:359] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.087858 19190 raft_consensus.cc:385] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:16.087918 19190 raft_consensus.cc:740] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ccde811b2d47d3b8296744839f3511, State: Initialized, Role: FOLLOWER
I20251023 02:09:16.088059 19190 consensus_queue.cc:260] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.088555 19190 ts_tablet_manager.cc:1434] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:09:16.192282 19195 raft_consensus.cc:493] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:16.192404 19195 raft_consensus.cc:515] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.193094 19195 leader_election.cc:290] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69357d9a54b3453281b4f959b241f47e (127.15.161.195:41191), 71ccde811b2d47d3b8296744839f3511 (127.15.161.193:45227)
I20251023 02:09:16.197481 19142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18f11726025f423da0c7cf5b5d72c229" candidate_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69357d9a54b3453281b4f959b241f47e" is_pre_election: true
I20251023 02:09:16.197487 18993 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18f11726025f423da0c7cf5b5d72c229" candidate_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ccde811b2d47d3b8296744839f3511" is_pre_election: true
I20251023 02:09:16.197661 19142 raft_consensus.cc:2468] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55b84bcd37a44538cbdc7698bb4e9f6 in term 0.
I20251023 02:09:16.197672 18993 raft_consensus.cc:2468] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55b84bcd37a44538cbdc7698bb4e9f6 in term 0.
I20251023 02:09:16.198006 19034 leader_election.cc:304] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 69357d9a54b3453281b4f959b241f47e, 71ccde811b2d47d3b8296744839f3511, a55b84bcd37a44538cbdc7698bb4e9f6; no voters:
I20251023 02:09:16.198230 19195 raft_consensus.cc:2804] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:16.198292 19195 raft_consensus.cc:493] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:16.198343 19195 raft_consensus.cc:3060] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:16.199712 19195 raft_consensus.cc:515] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.200086 19195 leader_election.cc:290] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [CANDIDATE]: Term 1 election: Requested vote from peers 69357d9a54b3453281b4f959b241f47e (127.15.161.195:41191), 71ccde811b2d47d3b8296744839f3511 (127.15.161.193:45227)
I20251023 02:09:16.200363 19142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18f11726025f423da0c7cf5b5d72c229" candidate_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69357d9a54b3453281b4f959b241f47e"
I20251023 02:09:16.200383 18993 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "18f11726025f423da0c7cf5b5d72c229" candidate_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ccde811b2d47d3b8296744839f3511"
I20251023 02:09:16.200513 18993 raft_consensus.cc:3060] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:16.200513 19142 raft_consensus.cc:3060] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:16.202618 19142 raft_consensus.cc:2468] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55b84bcd37a44538cbdc7698bb4e9f6 in term 1.
I20251023 02:09:16.202618 18993 raft_consensus.cc:2468] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55b84bcd37a44538cbdc7698bb4e9f6 in term 1.
I20251023 02:09:16.202968 19033 leader_election.cc:304] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71ccde811b2d47d3b8296744839f3511, a55b84bcd37a44538cbdc7698bb4e9f6; no voters:
I20251023 02:09:16.203154 19195 raft_consensus.cc:2804] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:16.203471 19195 raft_consensus.cc:697] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 1 LEADER]: Becoming Leader. State: Replica: a55b84bcd37a44538cbdc7698bb4e9f6, State: Running, Role: LEADER
I20251023 02:09:16.203682 19195 consensus_queue.cc:237] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } }
I20251023 02:09:16.205436 18890 catalog_manager.cc:5649] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a55b84bcd37a44538cbdc7698bb4e9f6 (127.15.161.194). New cstate: current_term: 1 leader_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a55b84bcd37a44538cbdc7698bb4e9f6" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 34961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:16.708982 19201 consensus_queue.cc:1048] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69357d9a54b3453281b4f959b241f47e" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 41191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:16.713209 19204 consensus_queue.cc:1048] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ccde811b2d47d3b8296744839f3511" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:16.816720 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:16.817422 18944 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:16.817732 18944 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:16.817826 18944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:17.818964 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:18.818230 18944 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:18.818588 18944 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:18.818673 18944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:18.820883 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:19.822829 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:20.819213 18944 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:20.819571 18944 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:20.819654 18944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:20.825062 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:21.827100 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:22.820010 18944 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:22.820350 18944 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:22.820448 18944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:22.828930 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:23.830947 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:24.820822 18944 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:24.821216 18944 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:24.821293 18944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:24.832908 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:25.835033 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:26.821594 18944 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:26.821955 18944 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:09:26.822031 18944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:26.837049 18939 placement_policy_util.cc:407] tablet 18f11726025f423da0c7cf5b5d72c229: detected majority of replicas (2 of 3) at location L0
I20251023 02:09:27.276985 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:09:27.291309 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:27.291576 16007 tablet_replica.cc:333] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511: stopping tablet replica
I20251023 02:09:27.291739 16007 raft_consensus.cc:2243] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:27.291893 16007 raft_consensus.cc:2272] T 18f11726025f423da0c7cf5b5d72c229 P 71ccde811b2d47d3b8296744839f3511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:27.305038 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:09:27.307468 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:09:27.315053 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:27.315289 16007 tablet_replica.cc:333] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6: stopping tablet replica
I20251023 02:09:27.315429 16007 raft_consensus.cc:2243] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:09:27.315703 16007 raft_consensus.cc:2272] T 18f11726025f423da0c7cf5b5d72c229 P a55b84bcd37a44538cbdc7698bb4e9f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:27.328523 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:09:27.330791 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:09:27.337535 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:09:27.337716 16007 tablet_replica.cc:333] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e: stopping tablet replica
I20251023 02:09:27.337881 16007 raft_consensus.cc:2243] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:27.338004 16007 raft_consensus.cc:2272] T 18f11726025f423da0c7cf5b5d72c229 P 69357d9a54b3453281b4f959b241f47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:27.350708 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:09:27.352934 16007 master.cc:561] Master@127.15.161.254:38515 shutting down...
I20251023 02:09:27.359494 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:09:27.359655 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:27.359798 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P c0a3341b82b94b06b3346a8a0b95e1ae: stopping tablet replica
I20251023 02:09:27.372656 16007 master.cc:583] Master@127.15.161.254:38515 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy (12605 ms)
[ RUN ] AutoRebalancerTest.TestMaxMovesPerServer
I20251023 02:09:27.381397 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:33133
I20251023 02:09:27.381798 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:27.383770 19212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:27.383937 19213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.384246 16007 server_base.cc:1047] running on GCE node
W20251023 02:09:27.384305 19215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.385027 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:27.385118 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:27.385164 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185367385148 us; error 0 us; skew 500 ppm
I20251023 02:09:27.385296 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:27.386245 16007 webserver.cc:492] Webserver started at http://127.15.161.254:43595/ using document root <none> and password file <none>
I20251023 02:09:27.386428 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:27.386493 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:27.386621 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:27.387130 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "5a15388250d243a49fbb6e7026c640fe"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.389114 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:09:27.390295 19220 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.390643 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:09:27.390724 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "5a15388250d243a49fbb6e7026c640fe"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.390810 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:27.397902 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:27.398304 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:27.412752 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:33133
I20251023 02:09:27.412786 19271 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:33133 every 8 connection(s)
I20251023 02:09:27.413735 19272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:27.417003 19272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe: Bootstrap starting.
I20251023 02:09:27.417910 19272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:27.419109 19272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe: No bootstrap required, opened a new log
I20251023 02:09:27.419510 19272 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a15388250d243a49fbb6e7026c640fe" member_type: VOTER }
I20251023 02:09:27.419620 19272 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:27.419653 19272 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a15388250d243a49fbb6e7026c640fe, State: Initialized, Role: FOLLOWER
I20251023 02:09:27.419771 19272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a15388250d243a49fbb6e7026c640fe" member_type: VOTER }
I20251023 02:09:27.419848 19272 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:09:27.419880 19272 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:09:27.419922 19272 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:27.421260 19272 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a15388250d243a49fbb6e7026c640fe" member_type: VOTER }
I20251023 02:09:27.421411 19272 leader_election.cc:304] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a15388250d243a49fbb6e7026c640fe; no voters:
I20251023 02:09:27.421691 19272 leader_election.cc:290] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:09:27.421787 19275 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:27.422117 19275 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a15388250d243a49fbb6e7026c640fe, State: Running, Role: LEADER
I20251023 02:09:27.422307 19275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a15388250d243a49fbb6e7026c640fe" member_type: VOTER }
I20251023 02:09:27.422361 19272 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:09:27.422984 19277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a15388250d243a49fbb6e7026c640fe. Latest consensus state: current_term: 1 leader_uuid: "5a15388250d243a49fbb6e7026c640fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a15388250d243a49fbb6e7026c640fe" member_type: VOTER } }
I20251023 02:09:27.423168 19277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [sys.catalog]: This master's current role is: LEADER
I20251023 02:09:27.423344 19276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a15388250d243a49fbb6e7026c640fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a15388250d243a49fbb6e7026c640fe" member_type: VOTER } }
I20251023 02:09:27.423499 19276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [sys.catalog]: This master's current role is: LEADER
I20251023 02:09:27.426721 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251023 02:09:27.427275 19291 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:09:27.427356 19291 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 02:09:27.427443 19280 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:09:27.428346 19280 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:09:27.430464 19280 catalog_manager.cc:1357] Generated new cluster ID: 1fb4c1a7636b49c4b1e89ca65038a892
I20251023 02:09:27.430538 19280 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:09:27.442935 19280 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:09:27.443542 19280 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:09:27.448732 19280 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe: Generated new TSK 0
I20251023 02:09:27.448947 19280 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:09:27.458990 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:27.461067 19293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:27.461125 19294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.461481 16007 server_base.cc:1047] running on GCE node
W20251023 02:09:27.461488 19296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.461760 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:27.461815 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:27.461838 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185367461835 us; error 0 us; skew 500 ppm
I20251023 02:09:27.461958 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:27.462978 16007 webserver.cc:492] Webserver started at http://127.15.161.193:42989/ using document root <none> and password file <none>
I20251023 02:09:27.463151 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:27.463202 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:27.463327 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:27.463799 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.465891 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:09:27.467162 19301 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.467494 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:27.467574 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.467664 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:27.479732 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:27.480072 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:27.480448 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:27.481071 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:27.481118 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.481176 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:27.481207 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.491550 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:43093
I20251023 02:09:27.491575 19363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:43093 every 8 connection(s)
I20251023 02:09:27.492832 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:27.494913 19368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:27.495504 19369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.496059 16007 server_base.cc:1047] running on GCE node
W20251023 02:09:27.496537 19371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.496812 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:27.496865 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:27.496919 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185367496915 us; error 0 us; skew 500 ppm
I20251023 02:09:27.497049 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:27.498143 16007 webserver.cc:492] Webserver started at http://127.15.161.194:39933/ using document root <none> and password file <none>
I20251023 02:09:27.498322 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:27.498417 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:27.498487 19364 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33133
I20251023 02:09:27.498517 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:27.498613 19364 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:27.498879 19364 heartbeater.cc:507] Master 127.15.161.254:33133 requested a full tablet report, sending...
I20251023 02:09:27.499023 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "6d761a2ec1dd4fed941acf04edae065f"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.499646 19237 ts_manager.cc:194] Registered new tserver with Master: 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:27.500837 19237 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43684
I20251023 02:09:27.501444 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:09:27.502627 19376 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.502889 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:27.502959 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "6d761a2ec1dd4fed941acf04edae065f"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.503038 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:27.509186 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:27.509488 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:27.509837 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:27.510442 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:27.510489 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.510546 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:27.510578 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.519558 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:36623
I20251023 02:09:27.519584 19438 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:36623 every 8 connection(s)
I20251023 02:09:27.520679 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:27.522711 19442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:27.523250 19445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:27.524542 19443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:27.525748 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:27.525965 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251023 02:09:27.525979 19439 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33133
W20251023 02:09:27.526062 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:27.526101 19439 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:27.526093 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185367526089 us; error 0 us; skew 500 ppm
I20251023 02:09:27.526257 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:27.526429 19439 heartbeater.cc:507] Master 127.15.161.254:33133 requested a full tablet report, sending...
I20251023 02:09:27.527139 19237 ts_manager.cc:194] Registered new tserver with Master: 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:27.527299 16007 webserver.cc:492] Webserver started at http://127.15.161.195:33391/ using document root <none> and password file <none>
I20251023 02:09:27.527494 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:27.527567 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:27.527654 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:27.528059 19237 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43688
I20251023 02:09:27.528254 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "42800175d61f4a9d8a28536807976ad6"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.530148 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20251023 02:09:27.531411 19450 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.531689 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:27.531761 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "42800175d61f4a9d8a28536807976ad6"
format_stamp: "Formatted at 2025-10-23 02:09:27 on dist-test-slave-bfbt"
I20251023 02:09:27.531860 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:27.545286 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:27.545596 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:27.545957 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:27.546597 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:27.546645 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.546705 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:27.546736 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:27.555763 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:38095
I20251023 02:09:27.555791 19512 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:38095 every 8 connection(s)
I20251023 02:09:27.560534 19513 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33133
I20251023 02:09:27.560631 19513 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:27.560885 19513 heartbeater.cc:507] Master 127.15.161.254:33133 requested a full tablet report, sending...
I20251023 02:09:27.561463 19237 ts_manager.cc:194] Registered new tserver with Master: 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:27.562006 19237 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43704
I20251023 02:09:27.562022 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005548845s
I20251023 02:09:28.502014 19364 heartbeater.cc:499] Master 127.15.161.254:33133 was elected leader, sending a full tablet report...
I20251023 02:09:28.529012 19439 heartbeater.cc:499] Master 127.15.161.254:33133 was elected leader, sending a full tablet report...
I20251023 02:09:28.563133 19513 heartbeater.cc:499] Master 127.15.161.254:33133 was elected leader, sending a full tablet report...
I20251023 02:09:28.588825 16007 test_util.cc:276] Using random seed: -330874352
I20251023 02:09:28.600051 19237 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43714:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu""\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:09:28.600652 19237 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:09:28.623274 19329 tablet_service.cc:1505] Processing CreateTablet for tablet 09457d2cf60844348195a77995f7cb88 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251023 02:09:28.623685 19325 tablet_service.cc:1505] Processing CreateTablet for tablet 4f894b15dfc24915a2e34e4dbdf85121 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251023 02:09:28.624020 19328 tablet_service.cc:1505] Processing CreateTablet for tablet 38f0952ea765453aaf13876360de864f (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251023 02:09:28.624262 19321 tablet_service.cc:1505] Processing CreateTablet for tablet 8ef6d3c01f1e49f386bb4898262f4d28 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251023 02:09:28.624754 19327 tablet_service.cc:1505] Processing CreateTablet for tablet 9973c11f67d54ed2a568e03d04d91ab2 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251023 02:09:28.625422 19326 tablet_service.cc:1505] Processing CreateTablet for tablet 24ed6c24dc924a0ab03d84241f9d96ea (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251023 02:09:28.625993 19324 tablet_service.cc:1505] Processing CreateTablet for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251023 02:09:28.626609 19323 tablet_service.cc:1505] Processing CreateTablet for tablet 5dab8a517d0a41ceb8ccf4b03d137571 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251023 02:09:28.627154 19322 tablet_service.cc:1505] Processing CreateTablet for tablet 579d6f2275814c5eae528d85a86bf288 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251023 02:09:28.627794 19320 tablet_service.cc:1505] Processing CreateTablet for tablet aece3449e9914e6a83067ce373506a58 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251023 02:09:28.623669 19329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09457d2cf60844348195a77995f7cb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.628331 19320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aece3449e9914e6a83067ce373506a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.628516 19324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f28c69fe3504243ad3cb29e0c5e2d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.630338 19322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579d6f2275814c5eae528d85a86bf288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.632480 19326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ed6c24dc924a0ab03d84241f9d96ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.633473 19327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9973c11f67d54ed2a568e03d04d91ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.636624 19322 tablet_service.cc:1505] Processing CreateTablet for tablet 390052bc00ba460a83d0c15f6f6b8808 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251023 02:09:28.637629 19322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 390052bc00ba460a83d0c15f6f6b8808. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.638242 19328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f0952ea765453aaf13876360de864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.638440 19323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab8a517d0a41ceb8ccf4b03d137571. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.637327 19324 tablet_service.cc:1505] Processing CreateTablet for tablet b746cb496f464e6c82421b1ab2dd1de0 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251023 02:09:28.642406 19324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b746cb496f464e6c82421b1ab2dd1de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.645200 19533 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.645210 19325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f894b15dfc24915a2e34e4dbdf85121. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.645597 19321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef6d3c01f1e49f386bb4898262f4d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.649655 19404 tablet_service.cc:1505] Processing CreateTablet for tablet 09457d2cf60844348195a77995f7cb88 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251023 02:09:28.649704 19403 tablet_service.cc:1505] Processing CreateTablet for tablet 38f0952ea765453aaf13876360de864f (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251023 02:09:28.650018 19403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f0952ea765453aaf13876360de864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.650180 19478 tablet_service.cc:1505] Processing CreateTablet for tablet 09457d2cf60844348195a77995f7cb88 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251023 02:09:28.650288 19476 tablet_service.cc:1505] Processing CreateTablet for tablet 9973c11f67d54ed2a568e03d04d91ab2 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251023 02:09:28.650547 19478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09457d2cf60844348195a77995f7cb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.650709 19476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9973c11f67d54ed2a568e03d04d91ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.651043 19477 tablet_service.cc:1505] Processing CreateTablet for tablet 38f0952ea765453aaf13876360de864f (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251023 02:09:28.651378 19477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f0952ea765453aaf13876360de864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.654549 19396 tablet_service.cc:1505] Processing CreateTablet for tablet 8ef6d3c01f1e49f386bb4898262f4d28 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251023 02:09:28.654979 19396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef6d3c01f1e49f386bb4898262f4d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.655066 19476 tablet_service.cc:1505] Processing CreateTablet for tablet 24ed6c24dc924a0ab03d84241f9d96ea (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251023 02:09:28.655442 19476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ed6c24dc924a0ab03d84241f9d96ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.655848 19399 tablet_service.cc:1505] Processing CreateTablet for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251023 02:09:28.656224 19399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f28c69fe3504243ad3cb29e0c5e2d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.658547 19398 tablet_service.cc:1505] Processing CreateTablet for tablet 5dab8a517d0a41ceb8ccf4b03d137571 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251023 02:09:28.658926 19398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab8a517d0a41ceb8ccf4b03d137571. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.654523 19534 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.660889 19478 tablet_service.cc:1505] Processing CreateTablet for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251023 02:09:28.661245 19478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f28c69fe3504243ad3cb29e0c5e2d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.661746 19534 tablet_bootstrap.cc:654] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.662505 19397 tablet_service.cc:1505] Processing CreateTablet for tablet 579d6f2275814c5eae528d85a86bf288 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251023 02:09:28.662904 19397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579d6f2275814c5eae528d85a86bf288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.665089 19395 tablet_service.cc:1505] Processing CreateTablet for tablet aece3449e9914e6a83067ce373506a58 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251023 02:09:28.665524 19395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aece3449e9914e6a83067ce373506a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.665553 19475 tablet_service.cc:1505] Processing CreateTablet for tablet 579d6f2275814c5eae528d85a86bf288 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251023 02:09:28.665887 19477 tablet_service.cc:1505] Processing CreateTablet for tablet 5dab8a517d0a41ceb8ccf4b03d137571 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251023 02:09:28.666015 19475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579d6f2275814c5eae528d85a86bf288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.666211 19477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab8a517d0a41ceb8ccf4b03d137571. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.654976 19401 tablet_service.cc:1505] Processing CreateTablet for tablet 24ed6c24dc924a0ab03d84241f9d96ea (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251023 02:09:28.669250 19401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ed6c24dc924a0ab03d84241f9d96ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.670455 19534 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.670579 19534 ts_tablet_manager.cc:1403] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20251023 02:09:28.671180 19534 raft_consensus.cc:359] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.671326 19534 raft_consensus.cc:385] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.671386 19534 raft_consensus.cc:740] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.654573 19402 tablet_service.cc:1505] Processing CreateTablet for tablet 9973c11f67d54ed2a568e03d04d91ab2 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251023 02:09:28.672626 19402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9973c11f67d54ed2a568e03d04d91ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.672864 19474 tablet_service.cc:1505] Processing CreateTablet for tablet 390052bc00ba460a83d0c15f6f6b8808 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251023 02:09:28.673259 19474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 390052bc00ba460a83d0c15f6f6b8808. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.673199 19475 tablet_service.cc:1505] Processing CreateTablet for tablet 8ef6d3c01f1e49f386bb4898262f4d28 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251023 02:09:28.673528 19475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef6d3c01f1e49f386bb4898262f4d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.673214 19476 tablet_service.cc:1505] Processing CreateTablet for tablet 4f894b15dfc24915a2e34e4dbdf85121 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251023 02:09:28.676050 19473 tablet_service.cc:1505] Processing CreateTablet for tablet b746cb496f464e6c82421b1ab2dd1de0 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251023 02:09:28.676144 19476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f894b15dfc24915a2e34e4dbdf85121. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.676415 19473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b746cb496f464e6c82421b1ab2dd1de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.655448 19400 tablet_service.cc:1505] Processing CreateTablet for tablet 4f894b15dfc24915a2e34e4dbdf85121 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251023 02:09:28.678496 19535 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.678645 19400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f894b15dfc24915a2e34e4dbdf85121. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.679703 19533 tablet_bootstrap.cc:654] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.680817 19477 tablet_service.cc:1505] Processing CreateTablet for tablet aece3449e9914e6a83067ce373506a58 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251023 02:09:28.681237 19477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aece3449e9914e6a83067ce373506a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.681620 19404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09457d2cf60844348195a77995f7cb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.684069 19396 tablet_service.cc:1505] Processing CreateTablet for tablet 390052bc00ba460a83d0c15f6f6b8808 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251023 02:09:28.684484 19396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 390052bc00ba460a83d0c15f6f6b8808. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.686195 19535 tablet_bootstrap.cc:654] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.689491 19533 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.689666 19533 ts_tablet_manager.cc:1403] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.000s
I20251023 02:09:28.690308 19533 raft_consensus.cc:359] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.690562 19533 raft_consensus.cc:385] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.690711 19533 raft_consensus.cc:740] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.690975 19533 consensus_queue.cc:260] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.691622 19533 ts_tablet_manager.cc:1434] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:28.691917 19533 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.693310 19533 tablet_bootstrap.cc:654] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.696605 19533 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.696776 19533 ts_tablet_manager.cc:1403] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20251023 02:09:28.696918 19402 tablet_service.cc:1505] Processing CreateTablet for tablet b746cb496f464e6c82421b1ab2dd1de0 (DEFAULT_TABLE table=test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251023 02:09:28.697430 19402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b746cb496f464e6c82421b1ab2dd1de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:28.697436 19533 raft_consensus.cc:359] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.697710 19533 raft_consensus.cc:385] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.697810 19533 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.698069 19533 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.700248 19533 ts_tablet_manager.cc:1434] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251023 02:09:28.702343 19533 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.703869 19533 tablet_bootstrap.cc:654] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.678256 19534 consensus_queue.cc:260] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.712141 19538 raft_consensus.cc:493] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.712277 19538 raft_consensus.cc:515] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.713435 19538 leader_election.cc:290] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:28.714056 19540 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.714262 19540 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.715843 19540 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.744558 19535 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.744745 19535 ts_tablet_manager.cc:1403] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.066s user 0.024s sys 0.004s
I20251023 02:09:28.745594 19535 raft_consensus.cc:359] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.745795 19535 raft_consensus.cc:385] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.745874 19535 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.746083 19535 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.751502 19535 ts_tablet_manager.cc:1434] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251023 02:09:28.751789 19535 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.753208 19535 tablet_bootstrap.cc:654] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.756652 19533 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.756866 19533 ts_tablet_manager.cc:1403] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.055s user 0.022s sys 0.025s
I20251023 02:09:28.756920 19535 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.757349 19534 ts_tablet_manager.cc:1434] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.087s user 0.021s sys 0.022s
I20251023 02:09:28.757417 19535 ts_tablet_manager.cc:1403] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251023 02:09:28.757644 19533 raft_consensus.cc:359] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.758296 19533 raft_consensus.cc:385] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.758368 19533 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.758562 19533 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.758956 19535 raft_consensus.cc:359] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.759166 19535 raft_consensus.cc:385] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.759246 19535 raft_consensus.cc:740] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.759661 19535 consensus_queue.cc:260] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.760174 19535 ts_tablet_manager.cc:1434] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251023 02:09:28.760447 19535 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.761041 19533 ts_tablet_manager.cc:1434] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251023 02:09:28.761307 19533 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.762143 19535 tablet_bootstrap.cc:654] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.762792 19533 tablet_bootstrap.cc:654] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.767114 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.767445 19414 raft_consensus.cc:2468] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:28.768059 19451 leader_election.cc:304] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.768085 19534 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.768504 19543 raft_consensus.cc:2804] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.768643 19543 raft_consensus.cc:493] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.768726 19543 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.770478 19534 tablet_bootstrap.cc:654] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.772010 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.772048 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
W20251023 02:09:28.772636 19302 leader_election.cc:343] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): Illegal state: must be running to vote when last-logged opid is not known
W20251023 02:09:28.772641 19452 leader_election.cc:343] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.773304 19543 raft_consensus.cc:515] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.774438 19533 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.774639 19533 ts_tablet_manager.cc:1403] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.002s
I20251023 02:09:28.774694 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:28.774880 19414 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.774296 19543 leader_election.cc:290] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:28.775584 19533 raft_consensus.cc:359] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.775699 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:28.775790 19533 raft_consensus.cc:385] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.775940 19533 raft_consensus.cc:740] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
W20251023 02:09:28.776051 19452 leader_election.cc:343] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.776225 19533 consensus_queue.cc:260] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.777232 19534 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.777380 19534 ts_tablet_manager.cc:1403] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20251023 02:09:28.777640 19414 raft_consensus.cc:2468] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:28.777985 19451 leader_election.cc:304] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters: 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:28.778038 19534 raft_consensus.cc:359] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.778211 19534 raft_consensus.cc:385] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.778292 19534 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.778412 19543 raft_consensus.cc:2804] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.778558 19543 raft_consensus.cc:697] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:28.778488 19534 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.778617 19533 ts_tablet_manager.cc:1434] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251023 02:09:28.778795 19543 consensus_queue.cc:237] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.779886 19535 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.780005 19535 ts_tablet_manager.cc:1403] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.008s
I20251023 02:09:28.780510 19533 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.780606 19535 raft_consensus.cc:359] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.780754 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.780754 19535 raft_consensus.cc:385] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.781594 19548 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.781740 19535 raft_consensus.cc:740] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.781747 19548 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.782240 19535 consensus_queue.cc:260] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.782778 19534 ts_tablet_manager.cc:1434] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
W20251023 02:09:28.782891 19302 leader_election.cc:343] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.783141 19533 tablet_bootstrap.cc:654] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.783105 19302 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb; no voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:28.783571 19540 raft_consensus.cc:2749] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:09:28.783568 19548 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.785928 19534 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.786790 19535 ts_tablet_manager.cc:1434] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20251023 02:09:28.789110 19535 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.789366 19534 tablet_bootstrap.cc:654] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.791256 19535 tablet_bootstrap.cc:654] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.801319 19237 catalog_manager.cc:5649] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:28.805430 19535 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.805774 19535 ts_tablet_manager.cc:1403] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.002s
I20251023 02:09:28.808071 19535 raft_consensus.cc:359] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.808343 19535 raft_consensus.cc:385] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.808427 19535 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.808617 19535 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.808983 19535 ts_tablet_manager.cc:1434] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251023 02:09:28.809198 19535 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.809914 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:28.810389 19533 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.810519 19533 ts_tablet_manager.cc:1403] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.017s
I20251023 02:09:28.810562 19535 tablet_bootstrap.cc:654] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
W20251023 02:09:28.811251 19378 leader_election.cc:343] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.811754 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.811807 19533 raft_consensus.cc:359] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.811961 19533 raft_consensus.cc:385] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.812036 19533 raft_consensus.cc:740] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
W20251023 02:09:28.812258 19377 leader_election.cc:343] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.812211 19533 consensus_queue.cc:260] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.812423 19377 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d761a2ec1dd4fed941acf04edae065f; no voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:28.812681 19533 ts_tablet_manager.cc:1434] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.813308 19556 raft_consensus.cc:2749] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:09:28.813392 19533 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.814523 19534 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.814656 19533 tablet_bootstrap.cc:654] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.814752 19534 ts_tablet_manager.cc:1403] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.009s
I20251023 02:09:28.815546 19534 raft_consensus.cc:359] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:28.815850 19534 raft_consensus.cc:385] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.815979 19534 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.816275 19534 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:28.816948 19534 ts_tablet_manager.cc:1434] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.817279 19534 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.819126 19534 tablet_bootstrap.cc:654] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.819679 19535 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.819885 19535 ts_tablet_manager.cc:1403] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20251023 02:09:28.820510 19535 raft_consensus.cc:359] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:28.820696 19535 raft_consensus.cc:385] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.820786 19535 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.821033 19535 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:28.821211 19534 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.821349 19534 ts_tablet_manager.cc:1403] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251023 02:09:28.821465 19535 ts_tablet_manager.cc:1434] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:28.821985 19535 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.821952 19534 raft_consensus.cc:359] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.822137 19534 raft_consensus.cc:385] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.822229 19534 raft_consensus.cc:740] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.822461 19534 consensus_queue.cc:260] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.823004 19534 ts_tablet_manager.cc:1434] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 02:09:28.823540 19535 tablet_bootstrap.cc:654] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.824298 19534 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.825937 19533 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.826099 19533 ts_tablet_manager.cc:1403] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20251023 02:09:28.826267 19534 tablet_bootstrap.cc:654] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.826778 19533 raft_consensus.cc:359] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.826915 19533 raft_consensus.cc:385] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.827000 19533 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.827180 19533 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.827633 19533 ts_tablet_manager.cc:1434] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:28.827965 19533 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.829747 19535 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.829742 19533 tablet_bootstrap.cc:654] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.829973 19535 ts_tablet_manager.cc:1403] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20251023 02:09:28.830721 19535 raft_consensus.cc:359] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.830961 19535 raft_consensus.cc:385] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.831076 19535 raft_consensus.cc:740] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.831352 19535 consensus_queue.cc:260] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.832600 19535 ts_tablet_manager.cc:1434] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 02:09:28.832913 19535 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.833536 19533 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.834117 19533 ts_tablet_manager.cc:1403] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251023 02:09:28.834146 19534 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.834492 19534 ts_tablet_manager.cc:1403] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20251023 02:09:28.834951 19533 raft_consensus.cc:359] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:28.835132 19535 tablet_bootstrap.cc:654] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.835201 19533 raft_consensus.cc:385] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.835294 19533 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.835397 19534 raft_consensus.cc:359] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.835628 19534 raft_consensus.cc:385] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.835556 19533 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:28.835727 19534 raft_consensus.cc:740] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.835968 19534 consensus_queue.cc:260] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.836046 19533 ts_tablet_manager.cc:1434] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:28.836447 19533 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.837009 19534 ts_tablet_manager.cc:1434] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.837303 19534 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.838446 19533 tablet_bootstrap.cc:654] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.838817 19534 tablet_bootstrap.cc:654] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.843477 19535 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.843641 19535 ts_tablet_manager.cc:1403] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20251023 02:09:28.843685 19533 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.843858 19533 ts_tablet_manager.cc:1403] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20251023 02:09:28.844419 19535 raft_consensus.cc:359] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.844601 19535 raft_consensus.cc:385] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.844566 19533 raft_consensus.cc:359] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.844682 19535 raft_consensus.cc:740] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.844746 19533 raft_consensus.cc:385] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.844854 19533 raft_consensus.cc:740] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.844988 19535 consensus_queue.cc:260] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.845045 19533 consensus_queue.cc:260] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.845482 19533 ts_tablet_manager.cc:1434] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:28.846292 19533 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.847225 19535 ts_tablet_manager.cc:1434] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251023 02:09:28.847509 19535 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.847813 19533 tablet_bootstrap.cc:654] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.849124 19535 tablet_bootstrap.cc:654] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.850051 19534 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.850231 19534 ts_tablet_manager.cc:1403] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20251023 02:09:28.850869 19534 raft_consensus.cc:359] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.851076 19534 raft_consensus.cc:385] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.851197 19534 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.851456 19534 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.852502 19534 ts_tablet_manager.cc:1434] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.852686 19533 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.852716 19535 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.852828 19533 ts_tablet_manager.cc:1403] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20251023 02:09:28.853346 19533 raft_consensus.cc:359] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.853462 19533 raft_consensus.cc:385] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.853538 19533 raft_consensus.cc:740] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.853708 19533 consensus_queue.cc:260] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.853816 19556 raft_consensus.cc:493] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.853924 19556 raft_consensus.cc:515] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.854179 19533 ts_tablet_manager.cc:1434] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:28.854434 19533 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.854691 19556 leader_election.cc:290] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.855767 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.855779 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
W20251023 02:09:28.856226 19377 leader_election.cc:343] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): Illegal state: must be running to vote when last-logged opid is not known
W20251023 02:09:28.856488 19378 leader_election.cc:343] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.852828 19534 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.856633 19378 leader_election.cc:304] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d761a2ec1dd4fed941acf04edae065f; no voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:28.856554 19533 tablet_bootstrap.cc:654] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.857560 19556 raft_consensus.cc:2749] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:09:28.858095 19534 tablet_bootstrap.cc:654] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.858481 19533 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.858579 19533 ts_tablet_manager.cc:1403] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 02:09:28.852829 19535 ts_tablet_manager.cc:1403] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20251023 02:09:28.859180 19533 raft_consensus.cc:359] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.859292 19533 raft_consensus.cc:385] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.859346 19533 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.859428 19535 raft_consensus.cc:359] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.859505 19533 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.859589 19535 raft_consensus.cc:385] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.859663 19535 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.859894 19533 ts_tablet_manager.cc:1434] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:28.859854 19535 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.860111 19533 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.860253 19535 ts_tablet_manager.cc:1434] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:28.860474 19535 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.862708 19534 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.862885 19534 ts_tablet_manager.cc:1403] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20251023 02:09:28.863126 19535 tablet_bootstrap.cc:654] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.863826 19540 raft_consensus.cc:493] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.863906 19533 tablet_bootstrap.cc:654] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.863965 19534 raft_consensus.cc:359] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.864151 19534 raft_consensus.cc:385] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.864240 19534 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.863965 19540 raft_consensus.cc:515] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.864477 19534 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.865069 19540 leader_election.cc:290] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.865628 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.865793 19414 raft_consensus.cc:2468] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:28.866235 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.866606 19302 leader_election.cc:304] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.866892 19540 raft_consensus.cc:2804] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.867000 19540 raft_consensus.cc:493] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.867112 19540 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
W20251023 02:09:28.867211 19302 leader_election.cc:343] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.867587 19533 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.867795 19533 ts_tablet_manager.cc:1403] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20251023 02:09:28.868398 19533 raft_consensus.cc:359] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.868527 19535 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.868607 19533 raft_consensus.cc:385] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.868664 19535 ts_tablet_manager.cc:1403] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20251023 02:09:28.868691 19533 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.868928 19533 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.869328 19535 raft_consensus.cc:359] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.869510 19535 raft_consensus.cc:385] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.869517 19533 ts_tablet_manager.cc:1434] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.869612 19535 raft_consensus.cc:740] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.869796 19533 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: Bootstrap starting.
I20251023 02:09:28.869839 19540 raft_consensus.cc:515] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.869884 19535 consensus_queue.cc:260] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.870371 19535 ts_tablet_manager.cc:1434] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.870432 19540 leader_election.cc:290] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.866621 19534 ts_tablet_manager.cc:1434] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251023 02:09:28.870679 19535 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.870826 19534 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.871400 19533 tablet_bootstrap.cc:654] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.872370 19535 tablet_bootstrap.cc:654] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.872982 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:28.873174 19414 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.873255 19534 tablet_bootstrap.cc:654] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.876293 19543 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.876377 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:28.876416 19543 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.876875 19543 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:28.877252 19540 raft_consensus.cc:493] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.877460 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:28.877465 19540 raft_consensus.cc:515] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.877661 19339 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:28.877987 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
W20251023 02:09:28.878186 19302 leader_election.cc:343] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): Illegal state: must be running to vote when last-logged opid is not known
I20251023 02:09:28.878240 19413 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:28.878453 19452 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:28.878865 19543 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.878978 19543 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.879072 19543 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.881742 19533 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: No bootstrap required, opened a new log
I20251023 02:09:28.881855 19533 ts_tablet_manager.cc:1403] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20251023 02:09:28.878221 19414 raft_consensus.cc:2468] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:28.882436 19533 raft_consensus.cc:359] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.882613 19533 raft_consensus.cc:385] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.882591 19302 leader_election.cc:304] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters: 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:28.882731 19533 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.882931 19533 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.883213 19543 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.883980 19543 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:28.884753 19540 leader_election.cc:290] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.884846 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:28.884972 19540 raft_consensus.cc:2804] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.885071 19339 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.885133 19540 raft_consensus.cc:697] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:28.885496 19540 consensus_queue.cc:237] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.888206 19339 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:28.888829 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.889102 19413 raft_consensus.cc:2393] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cfe6544d1b249e59e921b04cd1c5bcb in current term 1: Already voted for candidate 42800175d61f4a9d8a28536807976ad6 in this term.
I20251023 02:09:28.890738 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.891358 19452 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:28.891749 19302 leader_election.cc:304] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb; no voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:28.892238 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:28.892486 19414 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.894263 19543 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.894873 19543 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:28.895416 19543 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.897001 19414 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:28.898502 19577 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.900113 19237 catalog_manager.cc:5649] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:28.906283 19534 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.906040 19237 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:28.907668 19577 raft_consensus.cc:2749] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:09:28.908525 19534 ts_tablet_manager.cc:1403] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.011s
I20251023 02:09:28.909487 19534 raft_consensus.cc:359] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.909796 19534 raft_consensus.cc:385] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.909909 19534 raft_consensus.cc:740] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.910210 19534 consensus_queue.cc:260] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.910888 19534 ts_tablet_manager.cc:1434] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:28.911314 19534 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.913600 19534 tablet_bootstrap.cc:654] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.915431 19577 raft_consensus.cc:493] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.915678 19533 ts_tablet_manager.cc:1434] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: Time spent starting tablet: real 0.034s user 0.015s sys 0.019s
I20251023 02:09:28.915614 19577 raft_consensus.cc:515] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.915875 19535 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.916059 19535 ts_tablet_manager.cc:1403] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.045s user 0.017s sys 0.026s
I20251023 02:09:28.916875 19535 raft_consensus.cc:359] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.917074 19535 raft_consensus.cc:385] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.917189 19535 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.917279 19534 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.917413 19534 ts_tablet_manager.cc:1403] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251023 02:09:28.917449 19535 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.917515 19577 leader_election.cc:290] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.917837 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.917979 19535 ts_tablet_manager.cc:1434] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:28.918009 19414 raft_consensus.cc:2468] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:28.918150 19534 raft_consensus.cc:359] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.918294 19535 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.918256 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.918316 19534 raft_consensus.cc:385] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.918957 19488 raft_consensus.cc:2468] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:28.919138 19302 leader_election.cc:304] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.918941 19534 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.919390 19577 raft_consensus.cc:2804] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.919481 19577 raft_consensus.cc:493] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.919569 19577 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.919525 19534 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.920113 19534 ts_tablet_manager.cc:1434] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251023 02:09:28.920234 19535 tablet_bootstrap.cc:654] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.921372 19534 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.922008 19577 raft_consensus.cc:515] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.923012 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:28.923188 19413 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.923240 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:28.923393 19488 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.923921 19534 tablet_bootstrap.cc:654] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.925827 19413 raft_consensus.cc:2468] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:28.925890 19577 leader_election.cc:290] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.926182 19302 leader_election.cc:304] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.926443 19577 raft_consensus.cc:2804] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.926622 19577 raft_consensus.cc:697] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:28.926882 19577 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.925829 19488 raft_consensus.cc:2468] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:28.930001 19581 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.930649 19581 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.931258 19581 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.931588 19543 raft_consensus.cc:493] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.931674 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.931725 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.931833 19414 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:28.931741 19543 raft_consensus.cc:515] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.931948 19488 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:28.932181 19302 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.932340 19543 leader_election.cc:290] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:28.932646 19581 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.932762 19581 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.932839 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:28.933055 19339 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:28.933209 19535 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.933409 19535 ts_tablet_manager.cc:1403] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20251023 02:09:28.933375 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:28.933605 19414 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:28.933732 19452 leader_election.cc:304] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:28.932861 19581 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.933987 19543 raft_consensus.cc:2804] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.934154 19543 raft_consensus.cc:493] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.934293 19543 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.934799 19535 raft_consensus.cc:359] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.934994 19535 raft_consensus.cc:385] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.935107 19535 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.935385 19535 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.936086 19535 ts_tablet_manager.cc:1434] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:09:28.936380 19535 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: Bootstrap starting.
I20251023 02:09:28.936275 19236 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:28.937084 19543 raft_consensus.cc:515] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.937981 19535 tablet_bootstrap.cc:654] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.938382 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:28.938567 19339 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.940191 19535 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: No bootstrap required, opened a new log
I20251023 02:09:28.940327 19535 ts_tablet_manager.cc:1403] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 02:09:28.940958 19535 raft_consensus.cc:359] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.941097 19581 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.941175 19535 raft_consensus.cc:385] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.941257 19535 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.941277 19339 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:28.941571 19535 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.941758 19452 leader_election.cc:304] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:28.941825 19543 leader_election.cc:290] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:28.941937 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:28.942262 19414 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.942317 19554 raft_consensus.cc:2804] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.942476 19554 raft_consensus.cc:697] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:28.942745 19554 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.943097 19535 ts_tablet_manager.cc:1434] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20251023 02:09:28.944150 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:28.944360 19488 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.944684 19414 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:28.941987 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:28.945102 19413 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.945802 19302 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.946014 19577 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.946162 19577 raft_consensus.cc:697] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:28.946367 19577 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.946955 19488 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:28.947544 19413 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:28.948024 19236 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:28.949002 19581 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.949097 19534 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.949348 19534 ts_tablet_manager.cc:1403] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.003s
I20251023 02:09:28.950084 19534 raft_consensus.cc:359] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.950366 19534 raft_consensus.cc:385] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.950546 19534 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.950913 19534 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.950910 19237 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:28.951856 19534 ts_tablet_manager.cc:1434] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:28.952128 19534 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.953969 19534 tablet_bootstrap.cc:654] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.955498 19534 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.955608 19534 ts_tablet_manager.cc:1403] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20251023 02:09:28.956027 19534 raft_consensus.cc:359] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.956146 19534 raft_consensus.cc:385] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.956228 19534 raft_consensus.cc:740] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.956387 19534 consensus_queue.cc:260] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.956745 19534 ts_tablet_manager.cc:1434] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:28.957022 19534 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: Bootstrap starting.
I20251023 02:09:28.958436 19534 tablet_bootstrap.cc:654] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: Neither blocks nor log segments found. Creating new log.
I20251023 02:09:28.959916 19534 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: No bootstrap required, opened a new log
I20251023 02:09:28.960047 19534 ts_tablet_manager.cc:1403] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:09:28.960515 19534 raft_consensus.cc:359] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.960634 19534 raft_consensus.cc:385] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:09:28.960683 19534 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Initialized, Role: FOLLOWER
I20251023 02:09:28.960817 19534 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.961135 19534 ts_tablet_manager.cc:1434] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:28.990599 19556 raft_consensus.cc:493] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:28.990736 19556 raft_consensus.cc:515] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.991122 19556 leader_election.cc:290] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.991475 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aece3449e9914e6a83067ce373506a58" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:28.991494 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aece3449e9914e6a83067ce373506a58" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:28.991664 19488 raft_consensus.cc:2468] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 0.
I20251023 02:09:28.991662 19339 raft_consensus.cc:2468] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 0.
I20251023 02:09:28.992000 19378 leader_election.cc:304] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.992216 19556 raft_consensus.cc:2804] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:28.992293 19556 raft_consensus.cc:493] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:28.992383 19556 raft_consensus.cc:3060] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.994398 19556 raft_consensus.cc:515] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.994979 19556 leader_election.cc:290] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:28.995283 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aece3449e9914e6a83067ce373506a58" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:28.995358 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aece3449e9914e6a83067ce373506a58" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:28.995438 19339 raft_consensus.cc:3060] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.995477 19488 raft_consensus.cc:3060] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:28.996888 19339 raft_consensus.cc:2468] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 1.
I20251023 02:09:28.996937 19488 raft_consensus.cc:2468] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 1.
I20251023 02:09:28.997215 19377 leader_election.cc:304] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:28.997406 19556 raft_consensus.cc:2804] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:28.997516 19556 raft_consensus.cc:697] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:28.997705 19556 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:28.999464 19237 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:29.000130 19556 raft_consensus.cc:493] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:29.000262 19556 raft_consensus.cc:515] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.000815 19556 leader_election.cc:290] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:29.001053 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38f0952ea765453aaf13876360de864f" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:29.001200 19339 raft_consensus.cc:2468] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 0.
I20251023 02:09:29.001547 19378 leader_election.cc:304] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.001626 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38f0952ea765453aaf13876360de864f" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:29.001771 19488 raft_consensus.cc:2468] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 0.
I20251023 02:09:29.001832 19556 raft_consensus.cc:2804] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:29.001909 19556 raft_consensus.cc:493] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:29.002000 19556 raft_consensus.cc:3060] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.003381 19556 raft_consensus.cc:515] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.003738 19556 leader_election.cc:290] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:29.004040 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38f0952ea765453aaf13876360de864f" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:29.004098 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38f0952ea765453aaf13876360de864f" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:29.004220 19339 raft_consensus.cc:3060] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.004263 19488 raft_consensus.cc:3060] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.005887 19488 raft_consensus.cc:2468] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 1.
I20251023 02:09:29.005893 19339 raft_consensus.cc:2468] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 1.
I20251023 02:09:29.006208 19377 leader_election.cc:304] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.006433 19556 raft_consensus.cc:2804] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:29.006572 19556 raft_consensus.cc:697] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:29.011288 19556 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.013171 19237 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:29.017086 19577 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:29.017180 19577 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.017570 19577 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:29.017913 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6" is_pre_election: true
I20251023 02:09:29.017913 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:29.018091 19488 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:29.018091 19413 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 0.
I20251023 02:09:29.018436 19302 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.018677 19577 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:29.018779 19577 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:29.018870 19577 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.020345 19577 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.020823 19577 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:29.021106 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:29.021225 19413 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.021240 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:29.021389 19488 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.022693 19413 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:29.022994 19302 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.023162 19577 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:29.023202 19488 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 1.
I20251023 02:09:29.023262 19577 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:29.023428 19577 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.025799 19237 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:29.058465 19543 raft_consensus.cc:493] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:29.058575 19543 raft_consensus.cc:515] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.059026 19543 leader_election.cc:290] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:29.059378 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:29.059417 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:29.059566 19413 raft_consensus.cc:2393] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42800175d61f4a9d8a28536807976ad6 in current term 1: Already voted for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in this term.
I20251023 02:09:29.059864 19451 leader_election.cc:304] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6; no voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:29.060055 19543 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.061359 19543 raft_consensus.cc:2749] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 02:09:29.092622 19543 raft_consensus.cc:493] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:29.092710 19543 raft_consensus.cc:515] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.093070 19543 leader_election.cc:290] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:29.093364 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:29.093379 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:29.093504 19339 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:29.093523 19413 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:29.093797 19452 leader_election.cc:304] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:29.093972 19543 raft_consensus.cc:2804] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:29.094031 19543 raft_consensus.cc:493] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:29.094066 19543 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.095535 19543 raft_consensus.cc:515] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.095883 19543 leader_election.cc:290] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:29.096194 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:29.096194 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:29.096354 19339 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.096354 19413 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.098410 19339 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:29.098410 19413 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:29.098747 19452 leader_election.cc:304] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:29.098917 19543 raft_consensus.cc:2804] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:29.099020 19543 raft_consensus.cc:697] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:29.099187 19543 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.100855 19237 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:29.116375 19543 raft_consensus.cc:493] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:29.116477 19543 raft_consensus.cc:515] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:29.116883 19543 leader_election.cc:290] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:29.117136 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:29.117213 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:29.117247 19413 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:29.117353 19339 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:29.117599 19451 leader_election.cc:304] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.117806 19543 raft_consensus.cc:2804] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:29.117862 19543 raft_consensus.cc:493] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:29.117903 19543 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.119225 19543 raft_consensus.cc:515] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:29.119575 19543 leader_election.cc:290] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:29.119907 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:29.119907 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:29.120059 19413 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.120059 19339 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.122040 19339 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:29.122040 19413 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:29.122373 19451 leader_election.cc:304] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.122574 19543 raft_consensus.cc:2804] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:29.122676 19543 raft_consensus.cc:697] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:29.122869 19543 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:29.124670 19237 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:29.250809 19543 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:09:29.250916 19543 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.251402 19543 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:29.251657 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" is_pre_election: true
I20251023 02:09:29.251788 19339 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:29.251792 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f" is_pre_election: true
I20251023 02:09:29.251951 19413 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 0.
I20251023 02:09:29.252075 19452 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:29.252267 19543 raft_consensus.cc:2804] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:09:29.252326 19543 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:09:29.252391 19543 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.253793 19543 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.254271 19543 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:29.254591 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:29.254652 19413 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:29.254725 19339 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.254763 19413 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:09:29.256348 19413 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:29.256529 19339 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 1.
I20251023 02:09:29.256685 19451 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.256888 19543 raft_consensus.cc:2804] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:09:29.256982 19543 raft_consensus.cc:697] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:29.257158 19543 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.258914 19237 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:29.291002 19543 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.294486 19543 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.295176 19577 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.299136 19577 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.339047 19543 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.352981 19554 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.370580 19577 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.373219 19414 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.373602 19624 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.374017 19412 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.374511 19414 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.375228 19543 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.375284 19538 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.378814 19338 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.380959 19554 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.388325 19337 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.392030 19554 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.400782 19409 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.403133 19338 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.406626 19543 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.407893 19336 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.409438 19554 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.415354 19554 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:09:29.472365 19590 log.cc:927] Time spent T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20251023 02:09:29.473358 19487 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.474766 19411 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.482376 19581 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.482952 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:29.483148 19540 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251023 02:09:29.489912 19408 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.491087 19540 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.509362 19581 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251023 02:09:29.516989 19484 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.517604 19556 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.509198 19540 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20251023 02:09:29.590535 19556 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:09:29.591393 19339 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:09:29.609956 19556 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:09:29.634116 19652 log.cc:927] Time spent T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20251023 02:09:29.604111 19661 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
W20251023 02:09:29.687994 19649 log.cc:927] Time spent T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
W20251023 02:09:29.757289 19451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.161.194:36623, user_credentials={real_user=slave}} blocked reactor thread for 88275.3us
I20251023 02:09:29.759217 19631 mvcc.cc:204] Tried to move back new op lower bound from 7213815273535475712 to 7213815271416270848. Current Snapshot: MvccSnapshot[applied={T|T < 7213815273535475712}]
I20251023 02:09:29.776535 19483 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "09457d2cf60844348195a77995f7cb88"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:29.776683 19483 raft_consensus.cc:606] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:29.795010 19484 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:29.795174 19484 raft_consensus.cc:606] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:29.799113 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251023 02:09:29.801162 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:29.825922 19543 raft_consensus.cc:993] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:29.826018 19538 raft_consensus.cc:1081] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:29.832906 19408 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:29.836755 19408 raft_consensus.cc:493] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:29.838522 19408 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:29.848915 19408 raft_consensus.cc:515] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.850627 19414 raft_consensus.cc:1240] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Rejecting Update request from peer 42800175d61f4a9d8a28536807976ad6 for earlier term 1. Current term is 2. Ops: []
I20251023 02:09:29.853257 19408 leader_election.cc:290] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:29.860853 19483 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:29.860986 19483 raft_consensus.cc:3055] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:29.861039 19483 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:29.861202 19483 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.861378 19483 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:29.866379 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:29.866617 19336 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:29.868301 19624 raft_consensus.cc:993] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:29.868463 19624 raft_consensus.cc:1081] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:29.870900 19414 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "09457d2cf60844348195a77995f7cb88"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:29.871937 19414 raft_consensus.cc:493] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:29.872009 19414 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:29.874428 19483 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
W20251023 02:09:29.875739 19468 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Service unavailable: leader transfer in progress
W20251023 02:09:29.877394 19460 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Illegal state: replica 42800175d61f4a9d8a28536807976ad6 is not leader of this config: current role FOLLOWER
W20251023 02:09:29.880455 19461 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Illegal state: replica 42800175d61f4a9d8a28536807976ad6 is not leader of this config: current role FOLLOWER
I20251023 02:09:29.882687 19377 leader_election.cc:304] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.883422 19556 raft_consensus.cc:2804] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:29.886430 19414 raft_consensus.cc:515] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:29.894536 19414 leader_election.cc:290] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:29.895991 19556 raft_consensus.cc:697] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:29.904565 19556 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:29.911365 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
W20251023 02:09:29.930207 19459 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Service unavailable: leader transfer in progress
W20251023 02:09:29.934008 19313 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44396: Illegal state: replica 5cfe6544d1b249e59e921b04cd1c5bcb is not leader of this config: current role FOLLOWER
I20251023 02:09:29.936930 19339 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
W20251023 02:09:29.938638 19462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Service unavailable: leader transfer in progress
I20251023 02:09:29.943616 19336 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
W20251023 02:09:29.943673 19463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Service unavailable: leader transfer in progress
W20251023 02:09:29.946705 19312 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44396: Illegal state: replica 5cfe6544d1b249e59e921b04cd1c5bcb is not leader of this config: current role FOLLOWER
W20251023 02:09:29.946663 19315 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44396: Illegal state: replica 5cfe6544d1b249e59e921b04cd1c5bcb is not leader of this config: current role FOLLOWER
W20251023 02:09:29.947950 19317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44396: Illegal state: replica 5cfe6544d1b249e59e921b04cd1c5bcb is not leader of this config: current role FOLLOWER
W20251023 02:09:29.955792 19377 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.161.195:38095, user_credentials={real_user=slave}} blocked reactor thread for 50138.1us
I20251023 02:09:29.956296 19484 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:29.956506 19484 raft_consensus.cc:3055] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Stepping down as leader of term 1
W20251023 02:09:29.956573 19459 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Service unavailable: leader transfer in progress
I20251023 02:09:29.956611 19484 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:29.956879 19484 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:29.957180 19484 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
W20251023 02:09:29.957249 19462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Illegal state: replica 42800175d61f4a9d8a28536807976ad6 is not leader of this config: current role FOLLOWER
I20251023 02:09:29.955472 19233 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 1 to 2, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
W20251023 02:09:29.960721 19466 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Service unavailable: leader transfer in progress
W20251023 02:09:29.963280 19466 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Illegal state: replica 42800175d61f4a9d8a28536807976ad6 is not leader of this config: current role FOLLOWER
W20251023 02:09:29.964129 19463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37876: Illegal state: replica 42800175d61f4a9d8a28536807976ad6 is not leader of this config: current role FOLLOWER
I20251023 02:09:29.974910 19484 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:09:29.991909 19339 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:09:29.992414 19378 leader_election.cc:304] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:29.995569 19681 raft_consensus.cc:2804] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:29.995744 19681 raft_consensus.cc:697] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:29.999881 19681 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
W20251023 02:09:30.000860 19394 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35530: Service unavailable: leader is not yet ready
I20251023 02:09:30.005574 19338 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20251023 02:09:30.009575 19339 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20251023 02:09:30.010798 19680 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:09:30.011126 19556 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20251023 02:09:30.017724 19230 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 1 to 2, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } }
I20251023 02:09:30.019021 19484 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20251023 02:09:30.019837 19484 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20251023 02:09:30.021145 19680 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:09:30.021231 19556 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:09:30.589048 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:30.591497 19690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:30.591564 19689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:30.591913 19692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:30.592097 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:30.592386 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:30.592437 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:30.592470 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185370592467 us; error 0 us; skew 500 ppm
I20251023 02:09:30.592592 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:30.593894 16007 webserver.cc:492] Webserver started at http://127.15.161.196:37079/ using document root <none> and password file <none>
I20251023 02:09:30.594074 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:30.594182 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:30.594331 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:30.595083 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "e56d3da1a4934bc59b49951222c4ba5f"
format_stamp: "Formatted at 2025-10-23 02:09:30 on dist-test-slave-bfbt"
I20251023 02:09:30.597460 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 02:09:30.598735 19697 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.599004 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:30.599085 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "e56d3da1a4934bc59b49951222c4ba5f"
format_stamp: "Formatted at 2025-10-23 02:09:30 on dist-test-slave-bfbt"
I20251023 02:09:30.599165 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:30.612747 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:30.613088 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:30.613480 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:30.614185 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:30.614264 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.614344 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:30.614387 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.626842 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:34873
I20251023 02:09:30.626868 19759 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:34873 every 8 connection(s)
I20251023 02:09:30.628204 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:30.630412 19764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:30.630712 19765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:30.631848 19767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:30.632023 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:30.632373 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:09:30.632452 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:30.632478 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185370632475 us; error 0 us; skew 500 ppm
I20251023 02:09:30.632606 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:30.633476 19760 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33133
I20251023 02:09:30.633582 19760 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:30.633698 16007 webserver.cc:492] Webserver started at http://127.15.161.197:46757/ using document root <none> and password file <none>
I20251023 02:09:30.633890 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:30.633894 19760 heartbeater.cc:507] Master 127.15.161.254:33133 requested a full tablet report, sending...
I20251023 02:09:30.633944 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:30.634078 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:30.634689 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-4-root/instance:
uuid: "d24706c22e344a48b1f6715bd3115461"
format_stamp: "Formatted at 2025-10-23 02:09:30 on dist-test-slave-bfbt"
I20251023 02:09:30.634708 19233 ts_manager.cc:194] Registered new tserver with Master: e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:30.635432 19233 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43716
I20251023 02:09:30.636802 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 02:09:30.637919 19772 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.638196 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:30.638284 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-4-root
uuid: "d24706c22e344a48b1f6715bd3115461"
format_stamp: "Formatted at 2025-10-23 02:09:30 on dist-test-slave-bfbt"
I20251023 02:09:30.638384 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-4-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-4-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-4-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:30.644766 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:30.645167 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:30.645583 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:30.646193 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:30.646242 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.646302 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:30.646337 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.659833 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.197:46797
I20251023 02:09:30.659863 19834 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.197:46797 every 8 connection(s)
I20251023 02:09:30.661070 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:09:30.666047 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:30.666229 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:09:30.668607 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:09:30.668880 19835 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33133
I20251023 02:09:30.669056 19835 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:30.669369 16007 server_base.cc:1047] running on GCE node
I20251023 02:09:30.669632 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251023 02:09:30.669636 19835 heartbeater.cc:507] Master 127.15.161.254:33133 requested a full tablet report, sending...
W20251023 02:09:30.669700 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:09:30.669838 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185370669834 us; error 0 us; skew 500 ppm
I20251023 02:09:30.669975 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:09:30.670492 19233 ts_manager.cc:194] Registered new tserver with Master: d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:09:30.671447 19233 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43720
I20251023 02:09:30.672079 16007 webserver.cc:492] Webserver started at http://127.15.161.198:40389/ using document root <none> and password file <none>
I20251023 02:09:30.672250 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:09:30.672305 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:09:30.672405 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:09:30.672930 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-5-root/instance:
uuid: "4a65545dd26a4a018e7a9927513adaad"
format_stamp: "Formatted at 2025-10-23 02:09:30 on dist-test-slave-bfbt"
I20251023 02:09:30.675161 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20251023 02:09:30.676486 19848 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.676815 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:30.676898 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-5-root
uuid: "4a65545dd26a4a018e7a9927513adaad"
format_stamp: "Formatted at 2025-10-23 02:09:30 on dist-test-slave-bfbt"
I20251023 02:09:30.676973 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-5-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-5-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-5-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:09:30.679389 19338 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:30.681505 19414 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:30.681502 19485 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:30.682036 19540 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:30.682507 19540 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:30.684319 19540 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.684661 19485 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.690232 19233 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.693341 19414 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.696305 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:09:30.696779 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:09:30.712862 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:09:30.713845 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:09:30.713922 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:09:30.714013 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:09:30.714056 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20251023 02:09:30.718487 19305 consensus_peers.cc:597] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b746cb496f464e6c82421b1ab2dd1de0. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:30.720978 19414 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:30.722617 19338 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:30.722991 19485 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:30.723270 19686 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:30.723872 19686 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:30.726320 19688 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.730829 19338 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.731781 19485 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.737116 19233 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index -1 to 29, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.741777 19414 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:30.743755 19338 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:30.745222 19661 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:30.746114 19485 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:30.746646 19661 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:30.748291 19661 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.749315 19485 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
W20251023 02:09:30.751832 19380 consensus_peers.cc:597] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aece3449e9914e6a83067ce373506a58. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:30.751607 19338 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
W20251023 02:09:30.752017 19380 consensus_peers.cc:597] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f894b15dfc24915a2e34e4dbdf85121. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:30.755131 19233 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index -1 to 30, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.763952 19485 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:30.765995 19414 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:30.766397 19338 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:30.766460 19538 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:30.767980 19538 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:30.769229 19414 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.768561 19538 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
W20251023 02:09:30.773104 19454 consensus_peers.cc:597] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f28c69fe3504243ad3cb29e0c5e2d93. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:30.773938 19230 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index -1 to 28, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.774713 19338 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:30.788224 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.198:38041
I20251023 02:09:30.788311 19919 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.198:38041 every 8 connection(s)
I20251023 02:09:30.795594 19920 heartbeater.cc:344] Connected to a master server at 127.15.161.254:33133
I20251023 02:09:30.795707 19920 heartbeater.cc:461] Registering TS with master...
I20251023 02:09:30.796217 19920 heartbeater.cc:507] Master 127.15.161.254:33133 requested a full tablet report, sending...
I20251023 02:09:30.796849 19236 ts_manager.cc:194] Registered new tserver with Master: 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041)
I20251023 02:09:30.797642 19236 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43728
I20251023 02:09:31.080015 19923 ts_tablet_manager.cc:933] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: Initiating tablet copy from peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:31.080698 19923 tablet_copy_client.cc:323] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:43093
I20251023 02:09:31.085814 19349 tablet_copy_service.cc:140] P 5cfe6544d1b249e59e921b04cd1c5bcb: Received BeginTabletCopySession request for tablet b746cb496f464e6c82421b1ab2dd1de0 from peer e56d3da1a4934bc59b49951222c4ba5f ({username='slave'} at 127.0.0.1:44426)
I20251023 02:09:31.085947 19349 tablet_copy_service.cc:161] P 5cfe6544d1b249e59e921b04cd1c5bcb: Beginning new tablet copy session on tablet b746cb496f464e6c82421b1ab2dd1de0 from peer e56d3da1a4934bc59b49951222c4ba5f at {username='slave'} at 127.0.0.1:44426: session id = e56d3da1a4934bc59b49951222c4ba5f-b746cb496f464e6c82421b1ab2dd1de0
I20251023 02:09:31.088131 19349 tablet_copy_source_session.cc:215] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:31.089151 19923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b746cb496f464e6c82421b1ab2dd1de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:31.094000 19923 tablet_copy_client.cc:806] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:31.094206 19923 tablet_copy_client.cc:670] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:31.096134 19924 tablet_copy_client.cc:1025] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:31.268827 19925 ts_tablet_manager.cc:933] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Initiating tablet copy from peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:31.269315 19925 tablet_copy_client.cc:323] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:36623
I20251023 02:09:31.274896 19424 tablet_copy_service.cc:140] P 6d761a2ec1dd4fed941acf04edae065f: Received BeginTabletCopySession request for tablet 4f894b15dfc24915a2e34e4dbdf85121 from peer e56d3da1a4934bc59b49951222c4ba5f ({username='slave'} at 127.0.0.1:35550)
I20251023 02:09:31.275027 19424 tablet_copy_service.cc:161] P 6d761a2ec1dd4fed941acf04edae065f: Beginning new tablet copy session on tablet 4f894b15dfc24915a2e34e4dbdf85121 from peer e56d3da1a4934bc59b49951222c4ba5f at {username='slave'} at 127.0.0.1:35550: session id = e56d3da1a4934bc59b49951222c4ba5f-4f894b15dfc24915a2e34e4dbdf85121
I20251023 02:09:31.277197 19424 tablet_copy_source_session.cc:215] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:31.278259 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f894b15dfc24915a2e34e4dbdf85121. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:31.282290 19925 tablet_copy_client.cc:806] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:31.282464 19925 tablet_copy_client.cc:670] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:31.283547 19926 tablet_copy_client.cc:1025] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:31.295784 19928 ts_tablet_manager.cc:933] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Initiating tablet copy from peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:31.296241 19928 tablet_copy_client.cc:323] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:36623
I20251023 02:09:31.296674 19424 tablet_copy_service.cc:140] P 6d761a2ec1dd4fed941acf04edae065f: Received BeginTabletCopySession request for tablet aece3449e9914e6a83067ce373506a58 from peer e56d3da1a4934bc59b49951222c4ba5f ({username='slave'} at 127.0.0.1:35550)
I20251023 02:09:31.296780 19424 tablet_copy_service.cc:161] P 6d761a2ec1dd4fed941acf04edae065f: Beginning new tablet copy session on tablet aece3449e9914e6a83067ce373506a58 from peer e56d3da1a4934bc59b49951222c4ba5f at {username='slave'} at 127.0.0.1:35550: session id = e56d3da1a4934bc59b49951222c4ba5f-aece3449e9914e6a83067ce373506a58
I20251023 02:09:31.298734 19424 tablet_copy_source_session.cc:215] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:31.299535 19928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aece3449e9914e6a83067ce373506a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:31.303623 19928 tablet_copy_client.cc:806] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:31.303792 19928 tablet_copy_client.cc:670] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:31.305127 19929 tablet_copy_client.cc:1025] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:31.386222 19930 ts_tablet_manager.cc:933] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: Initiating tablet copy from peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:31.386699 19930 tablet_copy_client.cc:323] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:38095
I20251023 02:09:31.391784 19498 tablet_copy_service.cc:140] P 42800175d61f4a9d8a28536807976ad6: Received BeginTabletCopySession request for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 from peer e56d3da1a4934bc59b49951222c4ba5f ({username='slave'} at 127.0.0.1:37906)
I20251023 02:09:31.391913 19498 tablet_copy_service.cc:161] P 42800175d61f4a9d8a28536807976ad6: Beginning new tablet copy session on tablet 1f28c69fe3504243ad3cb29e0c5e2d93 from peer e56d3da1a4934bc59b49951222c4ba5f at {username='slave'} at 127.0.0.1:37906: session id = e56d3da1a4934bc59b49951222c4ba5f-1f28c69fe3504243ad3cb29e0c5e2d93
I20251023 02:09:31.393838 19498 tablet_copy_source_session.cc:215] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:31.394783 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f28c69fe3504243ad3cb29e0c5e2d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:31.399003 19930 tablet_copy_client.cc:806] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:31.399189 19930 tablet_copy_client.cc:670] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:31.400475 19931 tablet_copy_client.cc:1025] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:31.636343 19760 heartbeater.cc:499] Master 127.15.161.254:33133 was elected leader, sending a full tablet report...
I20251023 02:09:31.672866 19835 heartbeater.cc:499] Master 127.15.161.254:33133 was elected leader, sending a full tablet report...
I20251023 02:09:31.798727 19920 heartbeater.cc:499] Master 127.15.161.254:33133 was elected leader, sending a full tablet report...
I20251023 02:09:31.801448 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:31.804144 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:31.804302 19485 raft_consensus.cc:606] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:31.804713 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:31.804903 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:32.159466 19937 raft_consensus.cc:993] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:32.159624 19937 raft_consensus.cc:1081] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:32.160101 19336 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44374
I20251023 02:09:32.160269 19336 raft_consensus.cc:493] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:32.160357 19336 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:32.162278 19336 raft_consensus.cc:515] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:32.162822 19336 leader_election.cc:290] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 2 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:32.163038 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:32.163117 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:32.163182 19414 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:32.163204 19485 raft_consensus.cc:3055] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:32.163249 19485 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:32.163455 19485 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:32.163803 19485 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:32.165525 19414 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 2.
I20251023 02:09:32.165896 19302 leader_election.cc:304] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:32.166100 19933 raft_consensus.cc:2804] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:32.166091 19485 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 2.
I20251023 02:09:32.166244 19933 raft_consensus.cc:697] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:32.166425 19933 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:32.168433 19236 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 1 to 2, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 2 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:32.175323 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:32.175446 19336 raft_consensus.cc:606] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Received request to transfer leadership
I20251023 02:09:32.179308 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:32.179406 19336 raft_consensus.cc:606] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Received request to transfer leadership
W20251023 02:09:32.179642 19222 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 1f28c69fe3504243ad3cb29e0c5e2d93 already in progress [suppressed 10 similar messages]
I20251023 02:09:32.180323 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:32.180415 19336 raft_consensus.cc:606] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Received request to transfer leadership
W20251023 02:09:32.180716 19285 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 1f28c69fe3504243ad3cb29e0c5e2d93 already in progress
W20251023 02:09:32.180764 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 1f28c69fe3504243ad3cb29e0c5e2d93 already in progress
W20251023 02:09:32.599771 19305 consensus_peers.cc:597] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:32.603659 19412 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 02:09:32.604034 19933 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:32.616014 19934 raft_consensus.cc:993] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:32.616200 19933 raft_consensus.cc:1081] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:32.616780 19414 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35516
I20251023 02:09:32.618291 19414 raft_consensus.cc:493] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:32.618402 19414 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:32.620929 19414 raft_consensus.cc:515] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:32.622006 19414 leader_election.cc:290] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 3 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:32.622332 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:32.622473 19488 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:32.622548 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:32.622653 19336 raft_consensus.cc:3055] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:32.622706 19336 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:32.622903 19336 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:32.623148 19336 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:32.624809 19488 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 3.
I20251023 02:09:32.625349 19377 leader_election.cc:304] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:32.625484 19336 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 3.
I20251023 02:09:32.625581 19935 raft_consensus.cc:2804] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:32.625726 19935 raft_consensus.cc:697] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:32.625939 19935 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:32.628058 19236 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 2 to 3, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 3 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 02:09:33.086263 19380 consensus_peers.cc:597] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:33.096457 19923 tablet_copy_client.cc:538] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:33.099081 19923 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap starting.
I20251023 02:09:33.111336 19336 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 02:09:33.111737 19935 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:33.124827 19488 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 02:09:33.125381 19935 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:33.141170 19923 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=107 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:33.141634 19923 tablet_bootstrap.cc:492] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap complete.
I20251023 02:09:33.141952 19923 ts_tablet_manager.cc:1403] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: Time spent bootstrapping tablet: real 0.043s user 0.034s sys 0.009s
I20251023 02:09:33.142755 19923 raft_consensus.cc:359] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.142997 19923 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Initialized, Role: LEARNER
I20251023 02:09:33.143244 19923 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.143968 19923 ts_tablet_manager.cc:1434] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:09:33.144717 19349 tablet_copy_service.cc:342] P 5cfe6544d1b249e59e921b04cd1c5bcb: Request end of tablet copy session e56d3da1a4934bc59b49951222c4ba5f-b746cb496f464e6c82421b1ab2dd1de0 received from {username='slave'} at 127.0.0.1:44426
I20251023 02:09:33.144850 19349 tablet_copy_service.cc:434] P 5cfe6544d1b249e59e921b04cd1c5bcb: ending tablet copy session e56d3da1a4934bc59b49951222c4ba5f-b746cb496f464e6c82421b1ab2dd1de0 on tablet b746cb496f464e6c82421b1ab2dd1de0 with peer e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:33.283834 19925 tablet_copy_client.cc:538] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:33.286738 19925 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap starting.
I20251023 02:09:33.305408 19928 tablet_copy_client.cc:538] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:33.308451 19928 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap starting.
I20251023 02:09:33.321959 19925 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:33.322305 19925 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap complete.
I20251023 02:09:33.322503 19925 ts_tablet_manager.cc:1403] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Time spent bootstrapping tablet: real 0.036s user 0.033s sys 0.002s
I20251023 02:09:33.323215 19925 raft_consensus.cc:359] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.323454 19925 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Initialized, Role: LEARNER
I20251023 02:09:33.323628 19925 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.324074 19925 ts_tablet_manager.cc:1434] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:33.324649 19424 tablet_copy_service.cc:342] P 6d761a2ec1dd4fed941acf04edae065f: Request end of tablet copy session e56d3da1a4934bc59b49951222c4ba5f-4f894b15dfc24915a2e34e4dbdf85121 received from {username='slave'} at 127.0.0.1:35550
I20251023 02:09:33.324765 19424 tablet_copy_service.cc:434] P 6d761a2ec1dd4fed941acf04edae065f: ending tablet copy session e56d3da1a4934bc59b49951222c4ba5f-4f894b15dfc24915a2e34e4dbdf85121 on tablet 4f894b15dfc24915a2e34e4dbdf85121 with peer e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:33.342926 19928 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=108 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:33.343204 19928 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap complete.
I20251023 02:09:33.343401 19928 ts_tablet_manager.cc:1403] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Time spent bootstrapping tablet: real 0.035s user 0.028s sys 0.008s
I20251023 02:09:33.343953 19928 raft_consensus.cc:359] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.344125 19928 raft_consensus.cc:740] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Initialized, Role: LEARNER
I20251023 02:09:33.344267 19928 consensus_queue.cc:260] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.344719 19928 ts_tablet_manager.cc:1434] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:33.345940 19424 tablet_copy_service.cc:342] P 6d761a2ec1dd4fed941acf04edae065f: Request end of tablet copy session e56d3da1a4934bc59b49951222c4ba5f-aece3449e9914e6a83067ce373506a58 received from {username='slave'} at 127.0.0.1:35550
I20251023 02:09:33.346019 19424 tablet_copy_service.cc:434] P 6d761a2ec1dd4fed941acf04edae065f: ending tablet copy session e56d3da1a4934bc59b49951222c4ba5f-aece3449e9914e6a83067ce373506a58 on tablet aece3449e9914e6a83067ce373506a58 with peer e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:33.400859 19930 tablet_copy_client.cc:538] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:33.403664 19930 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap starting.
I20251023 02:09:33.423390 19734 raft_consensus.cc:1217] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 02:09:33.433555 19734 raft_consensus.cc:1217] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.29->[2.30-2.30] Dedup: 2.30->[]
I20251023 02:09:33.446017 19930 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=98 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:33.446312 19930 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap complete.
I20251023 02:09:33.446504 19930 ts_tablet_manager.cc:1403] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: Time spent bootstrapping tablet: real 0.043s user 0.035s sys 0.007s
I20251023 02:09:33.446985 19930 raft_consensus.cc:359] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.447129 19930 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Initialized, Role: LEARNER
I20251023 02:09:33.447299 19930 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.447678 19930 ts_tablet_manager.cc:1434] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:33.448158 19498 tablet_copy_service.cc:342] P 42800175d61f4a9d8a28536807976ad6: Request end of tablet copy session e56d3da1a4934bc59b49951222c4ba5f-1f28c69fe3504243ad3cb29e0c5e2d93 received from {username='slave'} at 127.0.0.1:37906
I20251023 02:09:33.448258 19498 tablet_copy_service.cc:434] P 42800175d61f4a9d8a28536807976ad6: ending tablet copy session e56d3da1a4934bc59b49951222c4ba5f-1f28c69fe3504243ad3cb29e0c5e2d93 on tablet 1f28c69fe3504243ad3cb29e0c5e2d93 with peer e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:33.510254 19734 raft_consensus.cc:1217] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 02:09:33.625304 19734 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Advancing to term 3
I20251023 02:09:33.627732 19734 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 3 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 02:09:33.628170 19688 consensus_queue.cc:1050] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:33.631481 19688 raft_consensus.cc:1064] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
I20251023 02:09:33.632071 19688 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 29, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:33.633733 19488 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:33.633909 19734 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 3 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:33.634778 19961 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:33.635049 19959 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:33.638286 19961 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 28 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.638959 19734 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 28 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.643085 19336 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:33.643537 19688 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:33.643738 19236 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 28 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:33.643851 19488 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 28 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.644634 19336 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 28 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.653761 19412 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:33.655886 19488 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 02:09:33.655886 19734 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 02:09:33.656320 19688 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:33.656456 19959 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:33.658147 19688 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Committing config change with OpId 3.32: config changed from index 31 to 32, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.658591 19734 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.662005 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:33.662698 19232 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 31 to 32, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New cstate: current_term: 3 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:33.662710 19488 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.666270 19326 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 with delete_type TABLET_DATA_TOMBSTONED (TS 5cfe6544d1b249e59e921b04cd1c5bcb not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:33.668434 19966 tablet_replica.cc:333] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:09:33.668648 19966 raft_consensus.cc:2243] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:33.668956 19966 raft_consensus.cc:2272] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:33.669637 19966 ts_tablet_manager.cc:1916] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:33.674103 19966 ts_tablet_manager.cc:1929] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.31
I20251023 02:09:33.674213 19966 log.cc:1199] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/1f28c69fe3504243ad3cb29e0c5e2d93
I20251023 02:09:33.674814 19222 catalog_manager.cc:4985] TS 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): tablet 1f28c69fe3504243ad3cb29e0c5e2d93 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:33.805279 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:33.825261 19688 raft_consensus.cc:1064] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
I20251023 02:09:33.826138 19688 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:33.828037 19336 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:33.828385 19734 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:33.828397 19488 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:33.828738 19961 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:33.828940 19960 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:33.829433 19960 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:33.833153 19960 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.833890 19336 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.833798 19734 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.836848 19488 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.838547 19230 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 29 to 30, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.840498 19414 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "09457d2cf60844348195a77995f7cb88"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:33.840624 19414 raft_consensus.cc:606] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:33.842201 19414 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:33.842320 19414 raft_consensus.cc:606] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:33.843294 19414 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:33.844913 19734 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:33.845206 19488 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:33.845682 19959 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:33.846112 19959 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:33.847697 19688 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.849164 19734 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.850044 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:33.850196 19336 raft_consensus.cc:606] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:33.850566 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 02:09:33.850742 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:33.851984 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet aece3449e9914e6a83067ce373506a58 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:33.852455 19232 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:33.852326 19488 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:33.856055 19326 tablet_service.cc:1552] Processing DeleteTablet for tablet aece3449e9914e6a83067ce373506a58 with delete_type TABLET_DATA_TOMBSTONED (TS 5cfe6544d1b249e59e921b04cd1c5bcb not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:33.856309 19966 tablet_replica.cc:333] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:09:33.856483 19966 raft_consensus.cc:2243] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:33.856715 19966 raft_consensus.cc:2272] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:33.857514 19966 ts_tablet_manager.cc:1916] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:33.861552 19966 ts_tablet_manager.cc:1929] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251023 02:09:33.861634 19966 log.cc:1199] T aece3449e9914e6a83067ce373506a58 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/aece3449e9914e6a83067ce373506a58
I20251023 02:09:33.862275 19222 catalog_manager.cc:4985] TS 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): tablet aece3449e9914e6a83067ce373506a58 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:33.902875 19974 raft_consensus.cc:1064] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
W20251023 02:09:33.902993 19974 raft_consensus.cc:1068] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Unable to promote non-voter e56d3da1a4934bc59b49951222c4ba5f: Service unavailable: leader transfer in progress
I20251023 02:09:33.908360 19961 raft_consensus.cc:993] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:33.908504 19959 raft_consensus.cc:1081] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:33.909171 19336 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44384
I20251023 02:09:33.909293 19336 raft_consensus.cc:493] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:33.909361 19336 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:33.909866 19960 raft_consensus.cc:1064] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
W20251023 02:09:33.909948 19960 raft_consensus.cc:1068] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: Unable to promote non-voter e56d3da1a4934bc59b49951222c4ba5f: Service unavailable: leader transfer in progress
I20251023 02:09:33.911593 19336 raft_consensus.cc:515] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.912110 19336 leader_election.cc:290] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:33.912812 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:33.913201 19414 raft_consensus.cc:3055] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:33.913259 19414 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:33.912830 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:33.913457 19488 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:33.913466 19414 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.913908 19414 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:33.915324 19488 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:33.915616 19414 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:33.915697 19302 leader_election.cc:304] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:33.915937 19975 raft_consensus.cc:2804] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:33.916055 19975 raft_consensus.cc:697] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:33.916312 19975 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:33.918886 19232 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 2 to 3, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 3 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:34.120317 19960 raft_consensus.cc:993] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:34.120458 19959 raft_consensus.cc:1081] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:34.121006 19488 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "09457d2cf60844348195a77995f7cb88"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:09:34.121169 19488 raft_consensus.cc:493] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:34.121268 19488 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:34.122876 19488 raft_consensus.cc:515] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:34.123342 19488 leader_election.cc:290] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:34.123580 19412 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:34.123690 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:34.123728 19412 raft_consensus.cc:3055] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:34.123798 19412 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:34.123800 19339 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:34.124027 19412 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:34.124310 19412 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:34.126889 19339 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 3.
I20251023 02:09:34.127231 19412 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 3.
I20251023 02:09:34.127292 19452 leader_election.cc:304] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:34.127482 19938 raft_consensus.cc:2804] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:34.127908 19938 raft_consensus.cc:697] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:34.128149 19938 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:34.130518 19232 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 2 to 3, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 3 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:34.299561 19975 raft_consensus.cc:1064] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
W20251023 02:09:34.299690 19975 raft_consensus.cc:1068] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: Unable to promote non-voter e56d3da1a4934bc59b49951222c4ba5f: Service unavailable: leader transfer in progress
I20251023 02:09:34.315860 19488 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:34.316258 19975 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:34.319448 19734 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEARNER]: Advancing to term 3
I20251023 02:09:34.321174 19734 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 LEARNER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:34.321601 19975 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:34.322139 19412 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:34.323374 19975 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:34.325634 19975 raft_consensus.cc:1064] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
I20251023 02:09:34.326190 19975 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 30, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:34.328379 19734 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 LEARNER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 02:09:34.328821 19974 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:34.335160 19485 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 02:09:34.336179 19985 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:34.338227 19975 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Committing config change with OpId 3.32: config changed from index 30 to 32, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.339107 19414 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 02:09:34.341361 19734 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 30 to 32, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.341934 19975 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:34.342006 19485 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 30 to 32, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.341979 19232 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 30 to 32, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:34.344627 19414 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 30 to 32, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.409209 19975 raft_consensus.cc:993] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:34.409320 19975 raft_consensus.cc:1081] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:34.409747 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37846
I20251023 02:09:34.409909 19485 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:34.409993 19485 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:34.411731 19485 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:34.412254 19485 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:34.412560 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:34.412575 19338 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:34.412775 19414 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:34.412791 19338 raft_consensus.cc:3055] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:34.412863 19338 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:34.413034 19338 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:34.413372 19338 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:34.414517 19414 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 2.
I20251023 02:09:34.414911 19451 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:34.415139 19938 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:34.415251 19938 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:34.415457 19938 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:34.415649 19338 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 2.
I20251023 02:09:34.417915 19233 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 1 to 2, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 2 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:34.584534 19339 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 02:09:34.585013 19938 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:34.588888 19414 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 02:09:34.589280 19938 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:34.910372 19339 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:34.911911 19734 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 02:09:34.911995 19414 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 02:09:34.912286 19990 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:34.912529 19985 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:34.914575 19975 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Committing config change with OpId 3.33: config changed from index 32 to 33, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.915056 19734 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.915315 19414 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 32 to 33, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:34.917254 19222 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4f894b15dfc24915a2e34e4dbdf85121 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:34.918536 19233 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 32 to 33, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New cstate: current_term: 3 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:34.922076 19473 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f894b15dfc24915a2e34e4dbdf85121 with delete_type TABLET_DATA_TOMBSTONED (TS 42800175d61f4a9d8a28536807976ad6 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:37842
I20251023 02:09:34.934347 19994 tablet_replica.cc:333] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:09:34.934523 19994 raft_consensus.cc:2243] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:34.934823 19994 raft_consensus.cc:2272] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:34.935546 19994 ts_tablet_manager.cc:1916] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:34.939023 19994 ts_tablet_manager.cc:1929] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.32
I20251023 02:09:34.939093 19994 log.cc:1199] T 4f894b15dfc24915a2e34e4dbdf85121 P 42800175d61f4a9d8a28536807976ad6: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/4f894b15dfc24915a2e34e4dbdf85121
I20251023 02:09:34.939738 19221 catalog_manager.cc:4985] TS 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): tablet 4f894b15dfc24915a2e34e4dbdf85121 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:35.007479 19414 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:35.007881 19938 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:35.011674 19338 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:35.012573 19996 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:35.013106 19734 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Advancing to term 2
I20251023 02:09:35.015228 19734 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:35.015657 19996 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:35.023512 19996 raft_consensus.cc:1064] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
I20251023 02:09:35.024281 19996 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:35.027063 19734 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 02:09:35.027175 19414 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 02:09:35.027468 19338 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 02:09:35.027551 19997 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:35.027709 19996 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:35.028051 19938 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:35.030203 19938 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 29 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.030807 19734 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.030803 19414 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.034878 19338 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.035759 19232 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 29 to 31, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.044620 19488 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:35.046061 19734 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:35.046272 19338 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:35.046595 19938 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:35.046805 19996 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:35.048660 19938 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.049260 19734 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.049260 19338 raft_consensus.cc:2955] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:35.051121 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet b746cb496f464e6c82421b1ab2dd1de0 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:35.052273 19233 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 31 to 32, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New cstate: current_term: 2 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:35.054912 19402 tablet_service.cc:1552] Processing DeleteTablet for tablet b746cb496f464e6c82421b1ab2dd1de0 with delete_type TABLET_DATA_TOMBSTONED (TS 6d761a2ec1dd4fed941acf04edae065f not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:35.056090 20003 tablet_replica.cc:333] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:09:35.056272 20003 raft_consensus.cc:2243] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:35.056551 20003 raft_consensus.cc:2272] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:35.057157 20003 ts_tablet_manager.cc:1916] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:35.060451 20003 ts_tablet_manager.cc:1929] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.31
I20251023 02:09:35.060523 20003 log.cc:1199] T b746cb496f464e6c82421b1ab2dd1de0 P 6d761a2ec1dd4fed941acf04edae065f: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/b746cb496f464e6c82421b1ab2dd1de0
I20251023 02:09:35.061065 19221 catalog_manager.cc:4985] TS 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): tablet b746cb496f464e6c82421b1ab2dd1de0 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:35.851117 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:35.853232 19338 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:35.853382 19338 raft_consensus.cc:606] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Received request to transfer leadership to TS e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:35.854180 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:35.854298 19485 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:35.855079 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:35.855196 19485 raft_consensus.cc:606] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:35.855563 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 02:09:35.855734 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:35.938274 19990 raft_consensus.cc:993] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower e56d3da1a4934bc59b49951222c4ba5f to start an election
I20251023 02:09:35.938380 19990 raft_consensus.cc:1081] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Signalling peer e56d3da1a4934bc59b49951222c4ba5f to start an election
I20251023 02:09:35.938808 19734 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121"
dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
from {username='slave'} at 127.0.0.1:52694
I20251023 02:09:35.938930 19734 raft_consensus.cc:493] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:35.938989 19734 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:35.940514 19734 raft_consensus.cc:515] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:35.941051 19734 leader_election.cc:290] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [CANDIDATE]: Term 4 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:35.941310 19338 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "e56d3da1a4934bc59b49951222c4ba5f" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:35.941445 19338 raft_consensus.cc:3055] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:35.941412 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f894b15dfc24915a2e34e4dbdf85121" candidate_uuid: "e56d3da1a4934bc59b49951222c4ba5f" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:35.941520 19338 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:35.941588 19414 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:35.941704 19338 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:35.941931 19338 raft_consensus.cc:3060] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:35.943243 19414 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d3da1a4934bc59b49951222c4ba5f in term 4.
I20251023 02:09:35.943499 19338 raft_consensus.cc:2468] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d3da1a4934bc59b49951222c4ba5f in term 4.
I20251023 02:09:35.943557 19698 leader_election.cc:304] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d761a2ec1dd4fed941acf04edae065f, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:35.944015 20005 raft_consensus.cc:2804] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:35.944324 20005 raft_consensus.cc:697] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 LEADER]: Becoming Leader. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Running, Role: LEADER
I20251023 02:09:35.944540 20005 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:35.946544 19232 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: term changed from 3 to 4, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196). New cstate: current_term: 4 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:36.056154 19338 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:36.058677 19414 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:36.059217 19485 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:36.059794 20004 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:36.060175 19990 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:36.070649 20004 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:36.072121 19414 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:36.073462 19485 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:36.076067 19338 consensus_queue.cc:237] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:36.078487 19232 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index -1 to 28, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
W20251023 02:09:36.079602 19304 consensus_peers.cc:597] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5dab8a517d0a41ceb8ccf4b03d137571. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:36.080194 19414 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251023 02:09:36.080591 19485 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251023 02:09:36.080935 20017 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251023 02:09:36.081499 20017 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251023 02:09:36.084687 19990 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:36.084945 19414 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:36.086097 19485 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
W20251023 02:09:36.087160 19303 consensus_peers.cc:597] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 390052bc00ba460a83d0c15f6f6b8808. This is attempt 1: this message will repeat every 5th retry.
W20251023 02:09:36.089143 19285 auto_rebalancer.cc:254] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
I20251023 02:09:36.089092 19233 catalog_manager.cc:5649] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index -1 to 27, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 02:09:36.093580 19285 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 8ef6d3c01f1e49f386bb4898262f4d28, TS 5cfe6544d1b249e59e921b04cd1c5bcb -> TS 4a65545dd26a4a018e7a9927513adaad move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
W20251023 02:09:36.093643 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 8ef6d3c01f1e49f386bb4898262f4d28, TS 5cfe6544d1b249e59e921b04cd1c5bcb -> TS 4a65545dd26a4a018e7a9927513adaad move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:36.100517 20024 raft_consensus.cc:993] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:36.100672 20024 raft_consensus.cc:1081] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
W20251023 02:09:36.100745 19285 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 4f894b15dfc24915a2e34e4dbdf85121, TS 6d761a2ec1dd4fed941acf04edae065f -> TS d24706c22e344a48b1f6715bd3115461 move failed, destination replica disappeared from tablet's Raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
W20251023 02:09:36.100826 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 4f894b15dfc24915a2e34e4dbdf85121, TS 6d761a2ec1dd4fed941acf04edae065f -> TS d24706c22e344a48b1f6715bd3115461 move failed, destination replica disappeared from tablet's Raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:36.101226 19414 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:36.101364 19414 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:36.101454 19414 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:36.103319 19414 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
W20251023 02:09:36.103821 19285 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 1f28c69fe3504243ad3cb29e0c5e2d93, TS 42800175d61f4a9d8a28536807976ad6 -> TS d24706c22e344a48b1f6715bd3115461 move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:36.103856 19414 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
W20251023 02:09:36.103886 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 1f28c69fe3504243ad3cb29e0c5e2d93, TS 42800175d61f4a9d8a28536807976ad6 -> TS d24706c22e344a48b1f6715bd3115461 move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:36.104100 19338 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:36.104161 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:36.104236 19338 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:36.104295 19485 raft_consensus.cc:3055] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:36.104355 19485 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:36.104558 19485 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:36.104876 19485 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:36.106400 19338 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:09:36.106750 19378 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:36.106981 20002 raft_consensus.cc:2804] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:36.107040 19485 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
W20251023 02:09:36.107555 19285 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet aece3449e9914e6a83067ce373506a58, TS 42800175d61f4a9d8a28536807976ad6 -> TS 4a65545dd26a4a018e7a9927513adaad move failed, destination replica disappeared from tablet's Raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
W20251023 02:09:36.107617 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet aece3449e9914e6a83067ce373506a58, TS 42800175d61f4a9d8a28536807976ad6 -> TS 4a65545dd26a4a018e7a9927513adaad move failed, destination replica disappeared from tablet's Raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:36.110292 20002 raft_consensus.cc:697] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:36.110546 20002 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:36.112687 19236 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 1 to 2, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:36.216004 20024 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:36.216205 19977 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:36.216715 19414 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:36.216866 19414 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:36.216944 19414 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:36.219090 19414 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:36.219597 19414 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:36.219857 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:36.219864 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:36.220003 19339 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:36.220021 19485 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:36.220070 19485 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:36.220284 19485 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:36.220522 19485 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:36.221542 19339 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:09:36.221825 19378 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:36.222046 20002 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:36.222183 20002 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:36.222162 19485 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:09:36.222397 20002 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:36.224355 19236 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 1 to 2, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:36.349637 19339 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:36.350077 20005 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:36.354827 19414 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:36.355764 20005 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:36.523732 20034 ts_tablet_manager.cc:933] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: Initiating tablet copy from peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:36.524305 20034 tablet_copy_client.cc:323] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:43093
I20251023 02:09:36.530843 19349 tablet_copy_service.cc:140] P 5cfe6544d1b249e59e921b04cd1c5bcb: Received BeginTabletCopySession request for tablet 390052bc00ba460a83d0c15f6f6b8808 from peer d24706c22e344a48b1f6715bd3115461 ({username='slave'} at 127.0.0.1:40250)
I20251023 02:09:36.530962 19349 tablet_copy_service.cc:161] P 5cfe6544d1b249e59e921b04cd1c5bcb: Beginning new tablet copy session on tablet 390052bc00ba460a83d0c15f6f6b8808 from peer d24706c22e344a48b1f6715bd3115461 at {username='slave'} at 127.0.0.1:40250: session id = d24706c22e344a48b1f6715bd3115461-390052bc00ba460a83d0c15f6f6b8808
I20251023 02:09:36.533365 19349 tablet_copy_source_session.cc:215] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:36.534387 20034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 390052bc00ba460a83d0c15f6f6b8808. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:36.538817 20034 tablet_copy_client.cc:806] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:36.539072 20034 tablet_copy_client.cc:670] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:36.540477 20035 tablet_copy_client.cc:1025] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:36.564625 19485 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 02:09:36.565026 20002 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:36.569396 19339 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 02:09:36.570375 20002 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:36.587005 20041 ts_tablet_manager.cc:933] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Initiating tablet copy from peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:36.587473 20041 tablet_copy_client.cc:323] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:43093
I20251023 02:09:36.593668 19349 tablet_copy_service.cc:140] P 5cfe6544d1b249e59e921b04cd1c5bcb: Received BeginTabletCopySession request for tablet 5dab8a517d0a41ceb8ccf4b03d137571 from peer 4a65545dd26a4a018e7a9927513adaad ({username='slave'} at 127.0.0.1:40260)
I20251023 02:09:36.593783 19349 tablet_copy_service.cc:161] P 5cfe6544d1b249e59e921b04cd1c5bcb: Beginning new tablet copy session on tablet 5dab8a517d0a41ceb8ccf4b03d137571 from peer 4a65545dd26a4a018e7a9927513adaad at {username='slave'} at 127.0.0.1:40260: session id = 4a65545dd26a4a018e7a9927513adaad-5dab8a517d0a41ceb8ccf4b03d137571
I20251023 02:09:36.596324 19349 tablet_copy_source_session.cc:215] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:36.597139 20041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab8a517d0a41ceb8ccf4b03d137571. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:36.602211 20041 tablet_copy_client.cc:806] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:36.602451 20041 tablet_copy_client.cc:670] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:36.603480 20042 tablet_copy_client.cc:1025] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:36.799206 19338 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 02:09:36.799630 20025 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:36.800658 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 02:09:36.801144 20025 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:37.856040 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:37.858769 19414 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:37.858948 19414 raft_consensus.cc:606] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:37.859812 19414 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "aece3449e9914e6a83067ce373506a58"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:37.859954 19414 raft_consensus.cc:606] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Received request to transfer leadership to TS e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:37.860303 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251023 02:09:37.860466 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:38.151237 20025 raft_consensus.cc:993] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower e56d3da1a4934bc59b49951222c4ba5f to start an election
I20251023 02:09:38.151391 20025 raft_consensus.cc:1081] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Signalling peer e56d3da1a4934bc59b49951222c4ba5f to start an election
I20251023 02:09:38.151906 19734 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "aece3449e9914e6a83067ce373506a58"
dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
from {username='slave'} at 127.0.0.1:52708
I20251023 02:09:38.152048 19734 raft_consensus.cc:493] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:38.152133 19734 raft_consensus.cc:3060] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:38.153762 19734 raft_consensus.cc:515] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:38.154362 19734 leader_election.cc:290] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [CANDIDATE]: Term 2 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:38.154673 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aece3449e9914e6a83067ce373506a58" candidate_uuid: "e56d3da1a4934bc59b49951222c4ba5f" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:38.154795 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aece3449e9914e6a83067ce373506a58" candidate_uuid: "e56d3da1a4934bc59b49951222c4ba5f" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:38.154845 19414 raft_consensus.cc:3055] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:38.154965 19485 raft_consensus.cc:3060] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:38.154973 19414 raft_consensus.cc:740] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:38.155264 19414 consensus_queue.cc:260] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:38.155524 19414 raft_consensus.cc:3060] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:38.156813 19485 raft_consensus.cc:2468] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d3da1a4934bc59b49951222c4ba5f in term 2.
I20251023 02:09:38.157130 19698 leader_election.cc:304] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:38.157209 19414 raft_consensus.cc:2468] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d3da1a4934bc59b49951222c4ba5f in term 2.
I20251023 02:09:38.157351 20047 raft_consensus.cc:2804] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:38.157912 20047 raft_consensus.cc:697] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEADER]: Becoming Leader. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Running, Role: LEADER
I20251023 02:09:38.158172 20047 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:38.160713 19236 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: term changed from 1 to 2, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196). New cstate: current_term: 2 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:38.227177 20051 raft_consensus.cc:993] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:38.227303 20051 raft_consensus.cc:1081] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:38.227821 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:09:38.227945 19485 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:38.228022 19485 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:38.230149 19485 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:38.230631 19485 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:38.230870 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:38.231015 19336 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:38.231029 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:38.231195 19414 raft_consensus.cc:3055] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:38.231254 19414 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:38.231429 19414 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:38.231721 19414 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:38.233072 19336 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 3.
I20251023 02:09:38.233377 19452 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:38.233577 19977 raft_consensus.cc:2804] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:38.233898 19977 raft_consensus.cc:697] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:38.233994 19414 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 3.
I20251023 02:09:38.234182 19977 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:38.236833 19236 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 2 to 3, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 3 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:38.540779 20034 tablet_copy_client.cc:538] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:38.543352 20034 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: Bootstrap starting.
I20251023 02:09:38.576745 20034 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=108 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:38.577018 20034 tablet_bootstrap.cc:492] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: Bootstrap complete.
I20251023 02:09:38.577219 20034 ts_tablet_manager.cc:1403] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: Time spent bootstrapping tablet: real 0.034s user 0.028s sys 0.004s
I20251023 02:09:38.577859 20034 raft_consensus.cc:359] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:38.578027 20034 raft_consensus.cc:740] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Initialized, Role: LEARNER
I20251023 02:09:38.578209 20034 consensus_queue.cc:260] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:38.578928 20034 ts_tablet_manager.cc:1434] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251023 02:09:38.579442 19349 tablet_copy_service.cc:342] P 5cfe6544d1b249e59e921b04cd1c5bcb: Request end of tablet copy session d24706c22e344a48b1f6715bd3115461-390052bc00ba460a83d0c15f6f6b8808 received from {username='slave'} at 127.0.0.1:40250
I20251023 02:09:38.579548 19349 tablet_copy_service.cc:434] P 5cfe6544d1b249e59e921b04cd1c5bcb: ending tablet copy session d24706c22e344a48b1f6715bd3115461-390052bc00ba460a83d0c15f6f6b8808 on tablet 390052bc00ba460a83d0c15f6f6b8808 with peer d24706c22e344a48b1f6715bd3115461
I20251023 02:09:38.603745 20041 tablet_copy_client.cc:538] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:38.606015 20041 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap starting.
I20251023 02:09:38.646365 20041 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:38.646678 20041 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap complete.
I20251023 02:09:38.646862 20041 ts_tablet_manager.cc:1403] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Time spent bootstrapping tablet: real 0.041s user 0.031s sys 0.011s
I20251023 02:09:38.647333 20041 raft_consensus.cc:359] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:38.647476 20041 raft_consensus.cc:740] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Initialized, Role: LEARNER
I20251023 02:09:38.647634 20041 consensus_queue.cc:260] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:38.648255 20041 ts_tablet_manager.cc:1434] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:38.648761 19349 tablet_copy_service.cc:342] P 5cfe6544d1b249e59e921b04cd1c5bcb: Request end of tablet copy session 4a65545dd26a4a018e7a9927513adaad-5dab8a517d0a41ceb8ccf4b03d137571 received from {username='slave'} at 127.0.0.1:40260
I20251023 02:09:38.648892 19349 tablet_copy_service.cc:434] P 5cfe6544d1b249e59e921b04cd1c5bcb: ending tablet copy session 4a65545dd26a4a018e7a9927513adaad-5dab8a517d0a41ceb8ccf4b03d137571 on tablet 5dab8a517d0a41ceb8ccf4b03d137571 with peer 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:38.699333 19414 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:38.699592 19485 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:38.699738 20047 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:38.700073 20047 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:38.771468 19336 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 02:09:38.771879 19977 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:38.775020 19414 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 02:09:38.775439 20072 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:38.906289 19810 raft_consensus.cc:1217] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251023 02:09:39.063895 19894 raft_consensus.cc:1217] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.28] Dedup: 1.28->[]
I20251023 02:09:39.345959 20077 raft_consensus.cc:1064] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
I20251023 02:09:39.346872 20077 consensus_queue.cc:237] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:39.348839 19810 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:39.349292 20076 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:39.349397 19414 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:39.349426 19485 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 02:09:39.349766 20079 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:39.350072 20077 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:39.352705 20077 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.353250 19485 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.356559 19810 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.356907 19236 catalog_manager.cc:5649] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 27 to 28, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:39.363941 19414 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.374102 19336 consensus_queue.cc:237] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:39.375577 19414 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:39.375946 19810 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:39.376013 20077 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:39.376483 20079 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:39.377892 20077 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.378417 19414 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.381506 19810 raft_consensus.cc:2955] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:39.382225 19222 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 390052bc00ba460a83d0c15f6f6b8808 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:39.382020 19233 catalog_manager.cc:5649] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 28 to 29, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:39.407126 19473 tablet_service.cc:1552] Processing DeleteTablet for tablet 390052bc00ba460a83d0c15f6f6b8808 with delete_type TABLET_DATA_TOMBSTONED (TS 42800175d61f4a9d8a28536807976ad6 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:37842
I20251023 02:09:39.407861 20087 tablet_replica.cc:333] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:09:39.408053 20087 raft_consensus.cc:2243] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:39.408401 20087 raft_consensus.cc:2272] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:39.409368 20087 ts_tablet_manager.cc:1916] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:39.414794 20087 ts_tablet_manager.cc:1929] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251023 02:09:39.414886 20087 log.cc:1199] T 390052bc00ba460a83d0c15f6f6b8808 P 42800175d61f4a9d8a28536807976ad6: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/390052bc00ba460a83d0c15f6f6b8808
I20251023 02:09:39.415498 19221 catalog_manager.cc:4985] TS 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): tablet 390052bc00ba460a83d0c15f6f6b8808 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:39.552603 20077 raft_consensus.cc:1064] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: attempting to promote NON_VOTER 4a65545dd26a4a018e7a9927513adaad to VOTER
I20251023 02:09:39.553429 20077 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:39.559362 19894 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:39.559831 20079 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:39.560830 19485 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:39.561167 19414 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:39.561666 20076 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:39.562065 20079 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:39.564633 20076 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.566460 19485 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.566515 19414 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.569093 19894 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.571413 19232 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 28 to 29, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.577745 19339 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:39.579483 19485 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:39.579685 19894 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:39.579879 20079 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:39.580955 20079 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:39.582057 20079 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.582798 19485 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.583616 19893 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:39.586289 19222 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 5dab8a517d0a41ceb8ccf4b03d137571 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:39.586391 19232 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 29 to 30, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:39.589877 19402 tablet_service.cc:1552] Processing DeleteTablet for tablet 5dab8a517d0a41ceb8ccf4b03d137571 with delete_type TABLET_DATA_TOMBSTONED (TS 6d761a2ec1dd4fed941acf04edae065f not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:39.591101 20093 tablet_replica.cc:333] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:09:39.591289 20093 raft_consensus.cc:2243] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:39.591610 20093 raft_consensus.cc:2272] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:39.592528 20093 ts_tablet_manager.cc:1916] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:39.596784 20093 ts_tablet_manager.cc:1929] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.29
I20251023 02:09:39.596876 20093 log.cc:1199] T 5dab8a517d0a41ceb8ccf4b03d137571 P 6d761a2ec1dd4fed941acf04edae065f: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/5dab8a517d0a41ceb8ccf4b03d137571
I20251023 02:09:39.597430 19221 catalog_manager.cc:4985] TS 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): tablet 5dab8a517d0a41ceb8ccf4b03d137571 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:39.860775 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:39.863066 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "09457d2cf60844348195a77995f7cb88"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:39.863219 19485 raft_consensus.cc:606] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:39.864073 19414 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:39.864179 19414 raft_consensus.cc:606] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:39.864943 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:39.865125 19336 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:39.866044 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:39.866205 19485 raft_consensus.cc:606] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:39.866556 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20251023 02:09:39.866727 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:39.964608 20077 raft_consensus.cc:993] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:39.964741 20077 raft_consensus.cc:1081] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:39.965268 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37846
I20251023 02:09:39.965386 19485 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:39.965481 19485 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:39.977244 19485 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:39.977691 19485 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:39.977975 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:39.978034 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:39.978139 19414 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:39.978137 19336 raft_consensus.cc:3055] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:39.978225 19336 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:39.978416 19336 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:39.978677 19336 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:39.980305 19414 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 2.
I20251023 02:09:39.980670 19336 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 2.
I20251023 02:09:39.980661 19451 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:39.980908 20075 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:39.981249 20075 raft_consensus.cc:697] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:39.981456 20075 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:39.983362 19232 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 1 to 2, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 2 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } }
I20251023 02:09:40.061254 20094 raft_consensus.cc:993] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:40.061349 20094 raft_consensus.cc:1081] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:40.061764 19414 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "09457d2cf60844348195a77995f7cb88"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:40.061889 19414 raft_consensus.cc:493] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:40.061968 19414 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:40.063521 19414 raft_consensus.cc:515] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:40.063953 19414 leader_election.cc:290] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 4 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:40.064286 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:40.064298 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09457d2cf60844348195a77995f7cb88" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:40.064419 19485 raft_consensus.cc:3055] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:40.064435 19336 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:40.064473 19485 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:40.064688 19485 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:40.064899 19485 raft_consensus.cc:3060] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:40.066087 19336 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 4.
I20251023 02:09:40.066442 19378 leader_election.cc:304] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:40.066586 19485 raft_consensus.cc:2468] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 4.
I20251023 02:09:40.066627 20051 raft_consensus.cc:2804] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:40.066947 20051 raft_consensus.cc:697] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:40.067140 20051 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } }
I20251023 02:09:40.068799 19232 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 3 to 4, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:40.093130 20096 raft_consensus.cc:993] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:40.093238 20096 raft_consensus.cc:1081] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:40.093647 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:09:40.093745 19485 raft_consensus.cc:493] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:40.093825 19485 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:40.095220 19485 raft_consensus.cc:515] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:40.095676 19485 leader_election.cc:290] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:40.095985 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:40.096071 19734 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:40.096129 19414 raft_consensus.cc:3055] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:40.096186 19734 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:40.096195 19414 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:40.096438 19414 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:40.096706 19414 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:40.098474 19734 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 4.
I20251023 02:09:40.098821 19454 leader_election.cc:304] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:40.098915 19414 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 4.
I20251023 02:09:40.099046 20094 raft_consensus.cc:2804] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:40.099184 20094 raft_consensus.cc:697] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:40.099355 20094 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:40.101138 19232 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 3 to 4, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 4 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:40.207612 20075 raft_consensus.cc:993] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:40.207707 20075 raft_consensus.cc:1081] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:40.208195 19336 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44374
I20251023 02:09:40.208307 19336 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:40.208391 19336 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:40.209939 19336 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:40.210394 19336 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:40.210656 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:40.210783 19485 raft_consensus.cc:3055] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:40.210745 19734 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:40.210851 19485 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:40.210924 19734 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:40.211032 19485 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:40.211261 19485 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:40.212412 19734 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:40.212673 19305 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:40.212896 20077 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:40.213037 20077 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:40.213042 19485 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:40.213241 20077 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:40.214979 19232 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 2 to 3, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 3 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:40.373957 19336 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251023 02:09:40.374342 20075 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:40.377164 19414 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251023 02:09:40.377578 20075 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 02:09:40.586769 19336 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 02:09:40.587210 20096 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:40.592056 19485 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 02:09:40.592520 20051 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:40.593868 19488 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:40.596302 19414 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:40.596721 20094 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:40.597043 19336 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:40.597427 20094 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:40.599345 19488 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 02:09:40.599831 20077 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:40.608192 20094 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index -1 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.608536 19414 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index -1 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.611747 19734 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 02:09:40.612071 19336 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index -1 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.612980 19735 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:40.612895 19230 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index -1 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New cstate: current_term: 3 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.613517 20075 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:40.613696 19734 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 33, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:40.615413 20079 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:40.623255 19336 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251023 02:09:40.624203 20067 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:40.624372 19414 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251023 02:09:40.625084 20067 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
W20251023 02:09:40.627717 19452 consensus_peers.cc:597] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ef6d3c01f1e49f386bb4898262f4d28. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:40.629833 19412 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:40.630474 20108 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
W20251023 02:09:40.642591 19700 consensus_peers.cc:597] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f894b15dfc24915a2e34e4dbdf85121. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:40.644120 20067 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 LEADER]: Committing config change with OpId 4.35: config changed from index 33 to 35, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.646111 19336 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 33 to 35, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.646420 19414 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 33 to 35, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.649091 19408 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:40.649175 19230 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 33 to 35, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New cstate: current_term: 4 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:40.652037 19336 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:40.652520 20126 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:40.653538 19488 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:40.654016 20096 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:40.654937 20126 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.655465 19336 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
W20251023 02:09:40.658319 19379 consensus_peers.cc:597] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38f0952ea765453aaf13876360de864f. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:40.659986 19232 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index -1 to 29, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.660497 19488 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:40.661104 19485 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.662742 19414 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
W20251023 02:09:40.662916 19452 consensus_peers.cc:597] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f28c69fe3504243ad3cb29e0c5e2d93. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:40.662725 19733 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:40.663240 20075 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:40.663607 20075 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:40.665691 20075 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 32 to 34, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.666692 19414 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 32 to 34, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.668181 19733 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 32 to 34, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.669716 19408 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:40.670405 19230 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 32 to 34, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New cstate: current_term: 4 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.679003 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:40.679261 19336 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:40.679385 20051 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:40.679709 20126 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
W20251023 02:09:40.685688 19378 consensus_peers.cc:597] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 579d6f2275814c5eae528d85a86bf288. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:40.690869 20096 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.691395 19336 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.691422 19485 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:40.694774 19232 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index -1 to 30, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:40.696311 19408 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
W20251023 02:09:40.698454 19379 consensus_peers.cc:597] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09457d2cf60844348195a77995f7cb88. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:40.698838 19485 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 02:09:40.698937 19336 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 02:09:40.699172 20051 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:40.699306 20126 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:40.700898 20096 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index -1 to 32, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.701349 19485 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index -1 to 32, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.701498 19336 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index -1 to 32, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:40.704329 19233 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index -1 to 32, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:41.030365 20133 ts_tablet_manager.cc:933] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: Initiating tablet copy from peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:41.031117 20133 tablet_copy_client.cc:323] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:38095
I20251023 02:09:41.040807 19498 tablet_copy_service.cc:140] P 42800175d61f4a9d8a28536807976ad6: Received BeginTabletCopySession request for tablet 8ef6d3c01f1e49f386bb4898262f4d28 from peer d24706c22e344a48b1f6715bd3115461 ({username='slave'} at 127.0.0.1:49684)
I20251023 02:09:41.040920 19498 tablet_copy_service.cc:161] P 42800175d61f4a9d8a28536807976ad6: Beginning new tablet copy session on tablet 8ef6d3c01f1e49f386bb4898262f4d28 from peer d24706c22e344a48b1f6715bd3115461 at {username='slave'} at 127.0.0.1:49684: session id = d24706c22e344a48b1f6715bd3115461-8ef6d3c01f1e49f386bb4898262f4d28
I20251023 02:09:41.043166 19498 tablet_copy_source_session.cc:215] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:41.044427 20133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef6d3c01f1e49f386bb4898262f4d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:41.046901 20134 ts_tablet_manager.cc:933] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: Initiating tablet copy from peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:41.047475 20134 tablet_copy_client.cc:323] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.196:34873
I20251023 02:09:41.051801 20133 tablet_copy_client.cc:806] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:41.052062 20133 tablet_copy_client.cc:670] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:41.053942 20135 tablet_copy_client.cc:1025] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:41.064324 19745 tablet_copy_service.cc:140] P e56d3da1a4934bc59b49951222c4ba5f: Received BeginTabletCopySession request for tablet 4f894b15dfc24915a2e34e4dbdf85121 from peer 4a65545dd26a4a018e7a9927513adaad ({username='slave'} at 127.0.0.1:60656)
I20251023 02:09:41.064495 19745 tablet_copy_service.cc:161] P e56d3da1a4934bc59b49951222c4ba5f: Beginning new tablet copy session on tablet 4f894b15dfc24915a2e34e4dbdf85121 from peer 4a65545dd26a4a018e7a9927513adaad at {username='slave'} at 127.0.0.1:60656: session id = 4a65545dd26a4a018e7a9927513adaad-4f894b15dfc24915a2e34e4dbdf85121
I20251023 02:09:41.066565 19745 tablet_copy_source_session.cc:215] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:41.067605 20134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f894b15dfc24915a2e34e4dbdf85121. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:41.072412 20134 tablet_copy_client.cc:806] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:41.072613 20134 tablet_copy_client.cc:670] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:41.073771 20138 tablet_copy_client.cc:1025] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:41.078658 20141 ts_tablet_manager.cc:933] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: Initiating tablet copy from peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:41.079649 20141 tablet_copy_client.cc:323] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:36623
I20251023 02:09:41.085323 19424 tablet_copy_service.cc:140] P 6d761a2ec1dd4fed941acf04edae065f: Received BeginTabletCopySession request for tablet 38f0952ea765453aaf13876360de864f from peer 4a65545dd26a4a018e7a9927513adaad ({username='slave'} at 127.0.0.1:52426)
I20251023 02:09:41.085467 19424 tablet_copy_service.cc:161] P 6d761a2ec1dd4fed941acf04edae065f: Beginning new tablet copy session on tablet 38f0952ea765453aaf13876360de864f from peer 4a65545dd26a4a018e7a9927513adaad at {username='slave'} at 127.0.0.1:52426: session id = 4a65545dd26a4a018e7a9927513adaad-38f0952ea765453aaf13876360de864f
I20251023 02:09:41.086326 20144 ts_tablet_manager.cc:933] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: Initiating tablet copy from peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:41.086993 20144 tablet_copy_client.cc:323] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:38095
I20251023 02:09:41.087486 19498 tablet_copy_service.cc:140] P 42800175d61f4a9d8a28536807976ad6: Received BeginTabletCopySession request for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 from peer d24706c22e344a48b1f6715bd3115461 ({username='slave'} at 127.0.0.1:49684)
I20251023 02:09:41.087589 19498 tablet_copy_service.cc:161] P 42800175d61f4a9d8a28536807976ad6: Beginning new tablet copy session on tablet 1f28c69fe3504243ad3cb29e0c5e2d93 from peer d24706c22e344a48b1f6715bd3115461 at {username='slave'} at 127.0.0.1:49684: session id = d24706c22e344a48b1f6715bd3115461-1f28c69fe3504243ad3cb29e0c5e2d93
I20251023 02:09:41.088049 19424 tablet_copy_source_session.cc:215] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:41.088932 20141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f0952ea765453aaf13876360de864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:41.089666 19498 tablet_copy_source_session.cc:215] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:41.090824 20144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f28c69fe3504243ad3cb29e0c5e2d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:41.094187 20141 tablet_copy_client.cc:806] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:41.094458 20141 tablet_copy_client.cc:670] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:41.095889 20142 tablet_copy_client.cc:1025] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:41.096374 20144 tablet_copy_client.cc:806] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:41.096591 20144 tablet_copy_client.cc:670] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:41.097893 20145 tablet_copy_client.cc:1025] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:41.157141 20146 ts_tablet_manager.cc:933] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: Initiating tablet copy from peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:41.157609 20146 tablet_copy_client.cc:323] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:36623
I20251023 02:09:41.161926 19424 tablet_copy_service.cc:140] P 6d761a2ec1dd4fed941acf04edae065f: Received BeginTabletCopySession request for tablet 579d6f2275814c5eae528d85a86bf288 from peer d24706c22e344a48b1f6715bd3115461 ({username='slave'} at 127.0.0.1:52432)
I20251023 02:09:41.162024 19424 tablet_copy_service.cc:161] P 6d761a2ec1dd4fed941acf04edae065f: Beginning new tablet copy session on tablet 579d6f2275814c5eae528d85a86bf288 from peer d24706c22e344a48b1f6715bd3115461 at {username='slave'} at 127.0.0.1:52432: session id = d24706c22e344a48b1f6715bd3115461-579d6f2275814c5eae528d85a86bf288
I20251023 02:09:41.164024 19424 tablet_copy_source_session.cc:215] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:41.164795 20146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579d6f2275814c5eae528d85a86bf288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:41.169047 20146 tablet_copy_client.cc:806] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:41.169239 20146 tablet_copy_client.cc:670] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:41.170380 20147 tablet_copy_client.cc:1025] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:41.298641 20148 ts_tablet_manager.cc:933] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: Initiating tablet copy from peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:41.299093 20148 tablet_copy_client.cc:323] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:36623
I20251023 02:09:41.299559 19424 tablet_copy_service.cc:140] P 6d761a2ec1dd4fed941acf04edae065f: Received BeginTabletCopySession request for tablet 09457d2cf60844348195a77995f7cb88 from peer 4a65545dd26a4a018e7a9927513adaad ({username='slave'} at 127.0.0.1:52426)
I20251023 02:09:41.299672 19424 tablet_copy_service.cc:161] P 6d761a2ec1dd4fed941acf04edae065f: Beginning new tablet copy session on tablet 09457d2cf60844348195a77995f7cb88 from peer 4a65545dd26a4a018e7a9927513adaad at {username='slave'} at 127.0.0.1:52426: session id = 4a65545dd26a4a018e7a9927513adaad-09457d2cf60844348195a77995f7cb88
I20251023 02:09:41.301405 19424 tablet_copy_source_session.cc:215] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:41.302312 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09457d2cf60844348195a77995f7cb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:41.306852 20148 tablet_copy_client.cc:806] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:41.307060 20148 tablet_copy_client.cc:670] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:41.308069 20149 tablet_copy_client.cc:1025] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:41.867002 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:41.869571 19488 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:41.869729 19488 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:41.870683 19414 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:41.870831 19414 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:41.871868 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:41.872016 19485 raft_consensus.cc:606] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:41.872864 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:41.873008 19336 raft_consensus.cc:606] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:41.873325 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20251023 02:09:41.873512 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:42.022576 20152 raft_consensus.cc:993] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:42.022754 20152 raft_consensus.cc:1081] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:42.023308 19336 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44374
I20251023 02:09:42.023458 19336 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:42.023564 19336 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.025782 19336 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.026350 19336 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 4 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:42.026703 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:42.026841 19488 raft_consensus.cc:3055] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:42.026901 19488 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:42.027112 19488 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.027489 19488 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.029426 19488 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 4.
I20251023 02:09:42.029659 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:42.029809 19414 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.029774 19302 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:42.030027 20153 raft_consensus.cc:2804] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:42.030169 20153 raft_consensus.cc:697] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:42.030361 20153 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.031963 19414 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 4.
I20251023 02:09:42.032991 19230 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 3 to 4, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 4 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:42.099785 20077 raft_consensus.cc:993] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:42.099917 20153 raft_consensus.cc:1081] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:42.100414 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37846
I20251023 02:09:42.100538 19485 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:42.100621 19485 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.102936 19485 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:42.103576 19485 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:42.103974 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:42.104130 19336 raft_consensus.cc:3055] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:42.104081 19733 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:42.104205 19336 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:42.104274 19733 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.104432 19336 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:42.104678 19336 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.106033 19733 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 4.
I20251023 02:09:42.106410 19336 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 4.
I20251023 02:09:42.106389 19454 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:42.106673 20152 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:42.106796 20152 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:42.106981 20152 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:42.109164 19230 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 3 to 4, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 4 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:42.119122 20096 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:42.119231 20154 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:42.119714 19336 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44384
I20251023 02:09:42.119832 19336 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:42.119910 19336 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:42.121272 19336 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.121680 19336 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:42.122045 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:42.122145 19414 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:42.122195 19485 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:42.122249 19414 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:42.122295 19414 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:42.122475 19414 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.122787 19414 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:42.124293 19485 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:42.124607 19302 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:42.124804 20153 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:42.124917 20153 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:42.125097 20153 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.126291 19414 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:42.127555 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 2 to 3, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 3 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:42.132172 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:42.132275 19336 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Received request to transfer leadership
I20251023 02:09:42.135635 20156 raft_consensus.cc:993] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:42.135756 20156 raft_consensus.cc:1081] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:42.136245 19414 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:42.136384 19414 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:42.136482 19414 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:42.138118 19414 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:42.138239 19336 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:42.138362 19336 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Received request to transfer leadership
I20251023 02:09:42.138583 19414 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 3 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
W20251023 02:09:42.138782 19222 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 579d6f2275814c5eae528d85a86bf288 already in progress
I20251023 02:09:42.138844 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:42.138891 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:42.138952 19336 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:42.138983 19485 raft_consensus.cc:3055] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:42.139034 19485 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:42.139230 19485 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:42.139458 19485 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:42.139634 19339 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:42.139761 19339 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Received request to transfer leadership
W20251023 02:09:42.140110 19285 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 579d6f2275814c5eae528d85a86bf288 already in progress
W20251023 02:09:42.140170 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 579d6f2275814c5eae528d85a86bf288 already in progress
I20251023 02:09:42.140830 19485 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 3.
I20251023 02:09:42.141038 19336 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 3.
I20251023 02:09:42.141217 19377 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:42.141417 20154 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:42.141526 20154 raft_consensus.cc:697] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:42.141690 20154 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:42.143541 19230 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 2 to 3, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 3 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:42.503108 19485 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 02:09:42.503536 20153 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:42.506492 19414 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
W20251023 02:09:42.506639 19303 consensus_peers.cc:597] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:42.507918 20077 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:42.548247 19336 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:42.548625 20166 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:42.551493 19734 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:42.551903 20166 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:42.555160 19414 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:42.555574 20077 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:42.563397 20169 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:42.563505 20169 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:42.564548 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
W20251023 02:09:42.564615 19303 consensus_peers.cc:597] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:42.564967 20169 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:42.565300 19406 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35516
I20251023 02:09:42.565464 19406 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:42.565546 19406 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.567883 19406 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.568639 19336 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:42.568779 19336 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:42.568836 19336 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:42.568882 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:42.569021 19336 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.569245 19336 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:42.569365 19406 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 4 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
W20251023 02:09:42.569531 20169 consensus_queue.cc:1175] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6d761a2ec1dd4fed941acf04edae065f" responder_term: 3 status { last_received { term: 3 index: 31 } last_committed_idx: 31 last_received_current_leader { term: 3 index: 31 } }
I20251023 02:09:42.571950 19336 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 4.
I20251023 02:09:42.573499 19378 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters: 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:42.573727 20154 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:42.573827 20154 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:42.574025 20154 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:42.575999 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 3 to 4, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:42.617825 19336 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251023 02:09:42.618232 20154 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:42.620998 19485 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251023 02:09:42.621366 20155 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251023 02:09:42.960155 19378 consensus_peers.cc:597] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:43.054241 20133 tablet_copy_client.cc:538] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:43.056914 20133 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: Bootstrap starting.
I20251023 02:09:43.074038 20134 tablet_copy_client.cc:538] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:43.076457 20134 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap starting.
I20251023 02:09:43.096230 20141 tablet_copy_client.cc:538] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:43.098151 20144 tablet_copy_client.cc:538] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:43.098871 20141 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: Bootstrap starting.
I20251023 02:09:43.100811 20144 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: Bootstrap starting.
I20251023 02:09:43.111955 20133 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:43.112322 20133 tablet_bootstrap.cc:492] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: Bootstrap complete.
I20251023 02:09:43.112601 20133 ts_tablet_manager.cc:1403] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: Time spent bootstrapping tablet: real 0.056s user 0.043s sys 0.001s
I20251023 02:09:43.113256 20133 raft_consensus.cc:359] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:43.113525 20133 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Initialized, Role: LEARNER
I20251023 02:09:43.113747 20133 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:43.116896 20133 ts_tablet_manager.cc:1434] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251023 02:09:43.117874 19498 tablet_copy_service.cc:342] P 42800175d61f4a9d8a28536807976ad6: Request end of tablet copy session d24706c22e344a48b1f6715bd3115461-8ef6d3c01f1e49f386bb4898262f4d28 received from {username='slave'} at 127.0.0.1:49684
I20251023 02:09:43.117981 19498 tablet_copy_service.cc:434] P 42800175d61f4a9d8a28536807976ad6: ending tablet copy session d24706c22e344a48b1f6715bd3115461-8ef6d3c01f1e49f386bb4898262f4d28 on tablet 8ef6d3c01f1e49f386bb4898262f4d28 with peer d24706c22e344a48b1f6715bd3115461
I20251023 02:09:43.122818 20134 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:43.123173 20134 tablet_bootstrap.cc:492] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap complete.
I20251023 02:09:43.123397 20134 ts_tablet_manager.cc:1403] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: Time spent bootstrapping tablet: real 0.047s user 0.044s sys 0.000s
I20251023 02:09:43.123993 20134 raft_consensus.cc:359] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:43.124186 20134 raft_consensus.cc:740] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Initialized, Role: LEARNER
I20251023 02:09:43.124372 20134 consensus_queue.cc:260] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:43.138486 20134 ts_tablet_manager.cc:1434] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: Time spent starting tablet: real 0.015s user 0.002s sys 0.011s
I20251023 02:09:43.138983 19745 tablet_copy_service.cc:342] P e56d3da1a4934bc59b49951222c4ba5f: Request end of tablet copy session 4a65545dd26a4a018e7a9927513adaad-4f894b15dfc24915a2e34e4dbdf85121 received from {username='slave'} at 127.0.0.1:60656
I20251023 02:09:43.139070 19745 tablet_copy_service.cc:434] P e56d3da1a4934bc59b49951222c4ba5f: ending tablet copy session 4a65545dd26a4a018e7a9927513adaad-4f894b15dfc24915a2e34e4dbdf85121 on tablet 4f894b15dfc24915a2e34e4dbdf85121 with peer 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:43.145328 19336 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 02:09:43.146289 20096 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:43.147075 19485 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:43.148623 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 02:09:43.149070 20155 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:43.152637 20144 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=98 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:43.153069 20144 tablet_bootstrap.cc:492] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: Bootstrap complete.
I20251023 02:09:43.153366 20144 ts_tablet_manager.cc:1403] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: Time spent bootstrapping tablet: real 0.053s user 0.038s sys 0.005s
I20251023 02:09:43.154053 20144 raft_consensus.cc:359] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:43.154285 20144 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Initialized, Role: LEARNER
I20251023 02:09:43.154501 20144 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:43.155113 20144 ts_tablet_manager.cc:1434] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:43.155980 19498 tablet_copy_service.cc:342] P 42800175d61f4a9d8a28536807976ad6: Request end of tablet copy session d24706c22e344a48b1f6715bd3115461-1f28c69fe3504243ad3cb29e0c5e2d93 received from {username='slave'} at 127.0.0.1:49684
I20251023 02:09:43.156114 19498 tablet_copy_service.cc:434] P 42800175d61f4a9d8a28536807976ad6: ending tablet copy session d24706c22e344a48b1f6715bd3115461-1f28c69fe3504243ad3cb29e0c5e2d93 on tablet 1f28c69fe3504243ad3cb29e0c5e2d93 with peer d24706c22e344a48b1f6715bd3115461
I20251023 02:09:43.169060 20141 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=121 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:43.169319 20141 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: Bootstrap complete.
I20251023 02:09:43.169508 20141 ts_tablet_manager.cc:1403] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: Time spent bootstrapping tablet: real 0.071s user 0.035s sys 0.000s
I20251023 02:09:43.169910 20141 raft_consensus.cc:359] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:43.170022 20141 raft_consensus.cc:740] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Initialized, Role: LEARNER
I20251023 02:09:43.170189 20141 consensus_queue.cc:260] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:43.170599 20141 ts_tablet_manager.cc:1434] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20251023 02:09:43.170851 20146 tablet_copy_client.cc:538] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:43.171087 19424 tablet_copy_service.cc:342] P 6d761a2ec1dd4fed941acf04edae065f: Request end of tablet copy session 4a65545dd26a4a018e7a9927513adaad-38f0952ea765453aaf13876360de864f received from {username='slave'} at 127.0.0.1:52426
I20251023 02:09:43.171180 19424 tablet_copy_service.cc:434] P 6d761a2ec1dd4fed941acf04edae065f: ending tablet copy session 4a65545dd26a4a018e7a9927513adaad-38f0952ea765453aaf13876360de864f on tablet 38f0952ea765453aaf13876360de864f with peer 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:43.173214 20146 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: Bootstrap starting.
I20251023 02:09:43.203771 20146 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=109 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:43.204043 20146 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: Bootstrap complete.
I20251023 02:09:43.204206 20146 ts_tablet_manager.cc:1403] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: Time spent bootstrapping tablet: real 0.031s user 0.026s sys 0.005s
I20251023 02:09:43.204708 20146 raft_consensus.cc:359] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:43.204859 20146 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Initialized, Role: LEARNER
I20251023 02:09:43.205008 20146 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:43.205338 20146 ts_tablet_manager.cc:1434] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:43.205865 19424 tablet_copy_service.cc:342] P 6d761a2ec1dd4fed941acf04edae065f: Request end of tablet copy session d24706c22e344a48b1f6715bd3115461-579d6f2275814c5eae528d85a86bf288 received from {username='slave'} at 127.0.0.1:52432
I20251023 02:09:43.205964 19424 tablet_copy_service.cc:434] P 6d761a2ec1dd4fed941acf04edae065f: ending tablet copy session d24706c22e344a48b1f6715bd3115461-579d6f2275814c5eae528d85a86bf288 on tablet 579d6f2275814c5eae528d85a86bf288 with peer d24706c22e344a48b1f6715bd3115461
I20251023 02:09:43.258857 19893 raft_consensus.cc:1217] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 02:09:43.308346 20148 tablet_copy_client.cc:538] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:43.310772 20148 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap starting.
I20251023 02:09:43.341885 20148 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=126 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:43.342204 20148 tablet_bootstrap.cc:492] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap complete.
I20251023 02:09:43.342427 20148 ts_tablet_manager.cc:1403] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: Time spent bootstrapping tablet: real 0.032s user 0.032s sys 0.000s
I20251023 02:09:43.343010 20148 raft_consensus.cc:359] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:43.343137 20148 raft_consensus.cc:740] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Initialized, Role: LEARNER
I20251023 02:09:43.343279 20148 consensus_queue.cc:260] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:43.343681 20148 ts_tablet_manager.cc:1434] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:43.344259 19424 tablet_copy_service.cc:342] P 6d761a2ec1dd4fed941acf04edae065f: Request end of tablet copy session 4a65545dd26a4a018e7a9927513adaad-09457d2cf60844348195a77995f7cb88 received from {username='slave'} at 127.0.0.1:52426
I20251023 02:09:43.344336 19424 tablet_copy_service.cc:434] P 6d761a2ec1dd4fed941acf04edae065f: ending tablet copy session 4a65545dd26a4a018e7a9927513adaad-09457d2cf60844348195a77995f7cb88 on tablet 09457d2cf60844348195a77995f7cb88 with peer 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:43.390414 19893 raft_consensus.cc:1217] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Deduplicated request from leader. Original: 4.34->[4.35-4.35] Dedup: 4.35->[]
I20251023 02:09:43.528697 19810 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 2 LEARNER]: Advancing to term 4
I20251023 02:09:43.530845 19810 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:43.531224 20096 consensus_queue.cc:1050] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:43.534017 20096 raft_consensus.cc:1064] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
I20251023 02:09:43.534705 20096 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 31, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:43.536327 19810 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:43.536393 19339 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:43.536573 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:43.536988 20155 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:43.537400 20155 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:43.537709 20155 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:43.539705 20154 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 30 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.540026 19336 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 30 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.542995 19809 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 30 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.547950 19485 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 30 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.548792 19233 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 30 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.558328 19406 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:43.560073 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:43.560092 19809 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:43.560606 20155 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:43.560849 20197 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:43.562361 20096 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.562880 19809 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.562908 19485 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:43.565649 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 579d6f2275814c5eae528d85a86bf288 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:43.566145 19236 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:43.567348 19809 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 3 LEARNER]: Advancing to term 4
I20251023 02:09:43.569319 19326 tablet_service.cc:1552] Processing DeleteTablet for tablet 579d6f2275814c5eae528d85a86bf288 with delete_type TABLET_DATA_TOMBSTONED (TS 5cfe6544d1b249e59e921b04cd1c5bcb not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:43.569617 19893 raft_consensus.cc:1217] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Deduplicated request from leader. Original: 4.31->[4.32-4.32] Dedup: 4.32->[]
I20251023 02:09:43.569895 20202 tablet_replica.cc:333] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:09:43.570137 20202 raft_consensus.cc:2243] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:43.570408 20202 raft_consensus.cc:2272] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:43.571728 20202 ts_tablet_manager.cc:1916] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:43.577639 20202 ts_tablet_manager.cc:1929] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.33
I20251023 02:09:43.577764 20202 log.cc:1199] T 579d6f2275814c5eae528d85a86bf288 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/579d6f2275814c5eae528d85a86bf288
I20251023 02:09:43.578351 19222 catalog_manager.cc:4985] TS 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): tablet 579d6f2275814c5eae528d85a86bf288 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:43.651352 19809 raft_consensus.cc:1217] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.33->[4.34-4.34] Dedup: 4.34->[]
I20251023 02:09:43.830291 20204 raft_consensus.cc:1064] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: attempting to promote NON_VOTER 4a65545dd26a4a018e7a9927513adaad to VOTER
I20251023 02:09:43.831002 20204 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 33, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:43.833042 19336 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251023 02:09:43.833061 19408 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251023 02:09:43.833174 19893 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251023 02:09:43.833606 20205 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:43.833847 20150 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:43.834197 20205 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:43.837692 20150 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 LEADER]: Committing config change with OpId 4.36: config changed from index 35 to 36, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.838225 19408 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.839202 19336 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.841320 19893 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.843334 19232 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 35 to 36, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.848831 19733 consensus_queue.cc:237] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 4.36, Last appended by leader: 33, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:43.850221 19893 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 4 index: 37. (index mismatch)
I20251023 02:09:43.850221 19408 raft_consensus.cc:1275] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 4 index: 37. (index mismatch)
I20251023 02:09:43.851333 20207 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.001s
I20251023 02:09:43.851600 20204 consensus_queue.cc:1048] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:09:43.853412 20207 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 LEADER]: Committing config change with OpId 4.37: config changed from index 36 to 37, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.853904 19408 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Committing config change with OpId 4.37: config changed from index 36 to 37, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.856773 19893 raft_consensus.cc:2955] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Committing config change with OpId 4.37: config changed from index 36 to 37, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.857129 19284 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4f894b15dfc24915a2e34e4dbdf85121 with cas_config_opid_index 36: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:43.858289 19230 catalog_manager.cc:5649] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 36 to 37, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New cstate: current_term: 4 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:43.861456 19326 tablet_service.cc:1552] Processing DeleteTablet for tablet 4f894b15dfc24915a2e34e4dbdf85121 with delete_type TABLET_DATA_TOMBSTONED (TS 5cfe6544d1b249e59e921b04cd1c5bcb not found in new config with opid_index 37) from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:43.861702 20202 tablet_replica.cc:333] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:09:43.861853 20202 raft_consensus.cc:2243] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:43.862030 20202 raft_consensus.cc:2272] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:43.862777 20202 ts_tablet_manager.cc:1916] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:43.867434 20202 ts_tablet_manager.cc:1929] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.36
I20251023 02:09:43.867568 20202 log.cc:1199] T 4f894b15dfc24915a2e34e4dbdf85121 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/4f894b15dfc24915a2e34e4dbdf85121
I20251023 02:09:43.868155 19222 catalog_manager.cc:4985] TS 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): tablet 4f894b15dfc24915a2e34e4dbdf85121 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:43.872994 20155 raft_consensus.cc:1064] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER 4a65545dd26a4a018e7a9927513adaad to VOTER
I20251023 02:09:43.873502 20155 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:43.873781 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:43.875661 19485 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:43.875965 19893 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:43.876339 20197 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:43.876627 20154 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:43.877417 19336 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:43.877890 19408 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:43.877861 20154 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:43.878100 19408 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:43.879660 19339 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:43.879840 19339 raft_consensus.cc:606] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Received request to transfer leadership to TS d24706c22e344a48b1f6715bd3115461
I20251023 02:09:43.881057 19408 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:09:43.881196 19408 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:43.881759 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 02:09:43.882047 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:43.891001 20155 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.891395 19488 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.891574 19336 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.894500 19893 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.894877 19236 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 29 to 30, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.899269 19408 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:43.900439 19336 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:43.900750 19893 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:43.900799 20096 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:43.901273 20096 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:43.902420 20096 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.904780 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 38f0952ea765453aaf13876360de864f with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:43.905092 19336 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.906214 19230 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 30 to 31, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:43.907402 19893 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:43.910257 19473 tablet_service.cc:1552] Processing DeleteTablet for tablet 38f0952ea765453aaf13876360de864f with delete_type TABLET_DATA_TOMBSTONED (TS 42800175d61f4a9d8a28536807976ad6 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:37842
I20251023 02:09:43.910876 20215 tablet_replica.cc:333] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:09:43.911026 20215 raft_consensus.cc:2243] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:43.911187 20215 raft_consensus.cc:2272] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:43.912215 20215 ts_tablet_manager.cc:1916] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:43.916280 20215 ts_tablet_manager.cc:1929] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251023 02:09:43.916381 20215 log.cc:1199] T 38f0952ea765453aaf13876360de864f P 42800175d61f4a9d8a28536807976ad6: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/38f0952ea765453aaf13876360de864f
I20251023 02:09:43.916981 19221 catalog_manager.cc:4985] TS 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): tablet 38f0952ea765453aaf13876360de864f (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:44.046061 20154 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:44.046240 20154 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:44.046893 19488 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:09:44.047040 19488 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:44.047132 19488 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:44.048599 19488 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.049162 19488 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:09:44.049430 19408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:44.049501 19809 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:09:44.049698 19408 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Stepping down as leader of term 4
I20251023 02:09:44.049721 19809 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:44.049767 19408 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:44.049938 19408 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.050253 19408 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:44.051666 19809 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 5.
I20251023 02:09:44.052024 19452 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, d24706c22e344a48b1f6715bd3115461; no voters:
I20251023 02:09:44.052129 19408 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 5.
I20251023 02:09:44.052244 20216 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20251023 02:09:44.052356 20216 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:44.052562 20216 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.054402 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 4 to 5, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 5 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:44.061259 20096 raft_consensus.cc:993] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:44.061407 20096 raft_consensus.cc:1081] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:44.061859 19338 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44384
I20251023 02:09:44.062023 19338 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:44.062157 19338 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:44.064111 19338 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:44.064774 19338 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 4 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:44.065007 19408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:44.065176 19408 raft_consensus.cc:3055] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Stepping down as leader of term 3
I20251023 02:09:44.065179 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:44.065271 19408 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:44.065363 19488 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:44.065510 19408 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:44.065795 19408 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Advancing to term 4
I20251023 02:09:44.067045 19488 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 4.
I20251023 02:09:44.067371 19302 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:44.067565 20188 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Leader election won for term 4
I20251023 02:09:44.067777 19408 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 4.
I20251023 02:09:44.067952 20188 raft_consensus.cc:697] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:44.068177 20188 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } }
I20251023 02:09:44.069972 19230 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 3 to 4, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 4 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:44.142513 20217 raft_consensus.cc:1064] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
I20251023 02:09:44.143189 20217 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 31, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.144881 19809 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:44.144937 19408 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:44.145370 20188 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:44.145658 20219 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:44.145984 19488 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:44.146661 20188 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:44.154131 20188 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.154528 19488 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.157714 19230 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.158655 19809 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.161806 19338 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.163445 19488 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:44.163918 20217 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:44.166415 20188 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.166741 19488 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.169790 19222 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 8ef6d3c01f1e49f386bb4898262f4d28 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:44.170753 19233 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 33 to 34, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New cstate: current_term: 4 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:44.171088 19810 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:44.171653 20219 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:44.172680 19810 raft_consensus.cc:2955] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.173911 19402 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ef6d3c01f1e49f386bb4898262f4d28 with delete_type TABLET_DATA_TOMBSTONED (TS 6d761a2ec1dd4fed941acf04edae065f not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:44.176613 20154 raft_consensus.cc:1064] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER 4a65545dd26a4a018e7a9927513adaad to VOTER
I20251023 02:09:44.177239 20154 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 30, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:44.178185 20225 tablet_replica.cc:333] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:09:44.178956 19893 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:44.179425 19338 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:44.179833 20197 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:44.180028 19488 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 02:09:44.180243 20225 raft_consensus.cc:2243] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:44.180244 20197 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:44.180464 20225 pending_rounds.cc:70] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Trying to abort 1 pending ops.
I20251023 02:09:44.180603 20096 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:44.180519 20225 pending_rounds.cc:77] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Aborting op as it isn't in flight: id { term: 4 index: 33 } timestamp: 7213815333448970240 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" old_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } } new_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } } }
I20251023 02:09:44.180811 20225 raft_consensus.cc:2917] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Aborting config change with OpId 4.33: Aborted: Op aborted
I20251023 02:09:44.180934 20225 raft_consensus.cc:2889] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20251023 02:09:44.181046 20225 raft_consensus.cc:2272] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:44.182930 20096 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.183346 19893 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.186834 20225 ts_tablet_manager.cc:1916] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:44.187911 19230 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad reported cstate change: config changed from index 32 to 33, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.188860 19338 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.190089 19485 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.192328 20225 ts_tablet_manager.cc:1929] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.33
I20251023 02:09:44.192473 20225 log.cc:1199] T 8ef6d3c01f1e49f386bb4898262f4d28 P 6d761a2ec1dd4fed941acf04edae065f: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/8ef6d3c01f1e49f386bb4898262f4d28
I20251023 02:09:44.194276 19221 catalog_manager.cc:4985] TS 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): tablet 8ef6d3c01f1e49f386bb4898262f4d28 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:44.195448 19406 consensus_queue.cc:237] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:44.196732 19893 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:44.196992 19338 raft_consensus.cc:1275] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 02:09:44.197156 20197 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:44.197440 20198 consensus_queue.cc:1048] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:44.199173 20198 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.199656 19893 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.202252 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 09457d2cf60844348195a77995f7cb88 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:44.202459 19338 raft_consensus.cc:2955] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:44.203301 19230 catalog_manager.cc:5649] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New cstate: current_term: 4 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:44.206236 19473 tablet_service.cc:1552] Processing DeleteTablet for tablet 09457d2cf60844348195a77995f7cb88 with delete_type TABLET_DATA_TOMBSTONED (TS 42800175d61f4a9d8a28536807976ad6 not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:37842
I20251023 02:09:44.206682 20215 tablet_replica.cc:333] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:09:44.206897 20215 raft_consensus.cc:2243] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:44.207095 20215 raft_consensus.cc:2272] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:44.207821 20215 ts_tablet_manager.cc:1916] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:44.212157 20215 ts_tablet_manager.cc:1929] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.33
I20251023 02:09:44.212316 20215 log.cc:1199] T 09457d2cf60844348195a77995f7cb88 P 42800175d61f4a9d8a28536807976ad6: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/09457d2cf60844348195a77995f7cb88
I20251023 02:09:44.213065 19221 catalog_manager.cc:4985] TS 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): tablet 09457d2cf60844348195a77995f7cb88 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:44.214686 20216 raft_consensus.cc:1064] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
I20251023 02:09:44.215269 20216 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 32, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.217284 19810 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251023 02:09:44.217504 19406 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251023 02:09:44.217702 19734 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251023 02:09:44.217725 20232 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:44.217949 20230 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:44.218252 20182 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:44.232234 20230 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Committing config change with OpId 4.35: config changed from index 34 to 35, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.232693 19734 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.236222 19810 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.236315 19406 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.239012 19236 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 34 to 35, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:44.244652 19485 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.245903 19734 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251023 02:09:44.246301 19810 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 4 index: 36. (index mismatch)
I20251023 02:09:44.246358 20182 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:44.247126 20232 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:44.248589 20230 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Committing config change with OpId 4.36: config changed from index 35 to 36, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.249024 19734 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.249506 19810 raft_consensus.cc:2955] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Committing config change with OpId 4.36: config changed from index 35 to 36, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:44.252374 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:44.252732 19232 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 35 to 36, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New cstate: current_term: 4 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:44.255132 19402 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f28c69fe3504243ad3cb29e0c5e2d93 with delete_type TABLET_DATA_TOMBSTONED (TS 6d761a2ec1dd4fed941acf04edae065f not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:44.255371 20225 tablet_replica.cc:333] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:09:44.255489 20225 raft_consensus.cc:2243] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:44.255704 20225 raft_consensus.cc:2272] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:44.256460 20225 ts_tablet_manager.cc:1916] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:44.273453 20225 ts_tablet_manager.cc:1929] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.35
I20251023 02:09:44.273532 20225 log.cc:1199] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 6d761a2ec1dd4fed941acf04edae065f: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/1f28c69fe3504243ad3cb29e0c5e2d93
I20251023 02:09:44.273990 19221 catalog_manager.cc:4985] TS 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): tablet 1f28c69fe3504243ad3cb29e0c5e2d93 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:44.392917 20221 raft_consensus.cc:993] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:44.393034 20221 raft_consensus.cc:1081] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Signalling peer d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:44.393471 19810 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808"
dest_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:54246
I20251023 02:09:44.393602 19810 raft_consensus.cc:493] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:44.393683 19810 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:44.395159 19810 raft_consensus.cc:515] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.395720 19810 leader_election.cc:290] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:44.396103 19406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:44.396091 19338 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:44.396262 19406 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:44.396287 19338 raft_consensus.cc:3055] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:44.396350 19338 raft_consensus.cc:740] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:44.396571 19338 consensus_queue.cc:260] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.396842 19338 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:44.397882 19406 raft_consensus.cc:2468] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 2.
I20251023 02:09:44.398242 19773 leader_election.cc:304] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d761a2ec1dd4fed941acf04edae065f, d24706c22e344a48b1f6715bd3115461; no voters:
I20251023 02:09:44.398458 19338 raft_consensus.cc:2468] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 2.
I20251023 02:09:44.398481 20224 raft_consensus.cc:2804] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:44.398874 20224 raft_consensus.cc:697] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 LEADER]: Becoming Leader. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Running, Role: LEADER
I20251023 02:09:44.399098 20224 consensus_queue.cc:237] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:44.400825 19233 catalog_manager.cc:5649] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 reported cstate change: term changed from 1 to 2, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to d24706c22e344a48b1f6715bd3115461 (127.15.161.197). New cstate: current_term: 2 leader_uuid: "d24706c22e344a48b1f6715bd3115461" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:44.478085 19406 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 02:09:44.478489 20232 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:44.482337 19810 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 02:09:44.482841 20230 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:44.491307 19406 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251023 02:09:44.491698 20221 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:44.498911 19488 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251023 02:09:44.499310 20188 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:44.859982 19338 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:44.860365 20224 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:44.862970 19406 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 02:09:44.863615 20245 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:45.257612 19338 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 4.30, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:45.259158 19488 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 02:09:45.259236 19406 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
W20251023 02:09:45.259332 19304 consensus_peers.cc:597] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 24ed6c24dc924a0ab03d84241f9d96ea. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:45.259672 20188 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:45.259861 20221 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:45.261490 20219 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Committing config change with OpId 4.31: config changed from index -1 to 31, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:45.262001 19488 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index -1 to 31, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:45.261955 19406 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index -1 to 31, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:45.265957 19233 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index -1 to 31, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New cstate: current_term: 4 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:45.266642 19406 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:45.268390 19893 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 02:09:45.268803 20198 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
W20251023 02:09:45.268986 19380 consensus_peers.cc:597] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38f0952ea765453aaf13876360de864f. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:45.271487 20198 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:45.272163 19893 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:45.275985 19338 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 02:09:45.276413 20251 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:45.278729 19338 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:45.280508 19230 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 31 to 32, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:45.283027 19339 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:45.285954 19488 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:45.286375 19894 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
W20251023 02:09:45.286536 19303 consensus_peers.cc:597] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5dab8a517d0a41ceb8ccf4b03d137571. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:45.286684 20219 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:45.286895 20221 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:45.289153 20188 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:45.289592 19893 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:45.289592 19488 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:45.292273 19230 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 30 to 31, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New cstate: current_term: 1 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:45.292842 19485 consensus_queue.cc:237] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:45.295977 19339 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:45.296329 19406 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 02:09:45.296422 20248 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251023 02:09:45.296763 19452 consensus_peers.cc:597] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 -> Peer d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797): Couldn't send request to peer d24706c22e344a48b1f6715bd3115461. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9973c11f67d54ed2a568e03d04d91ab2. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:45.296885 20248 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 02:09:45.298700 20247 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:45.299438 19406 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:45.301678 19339 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } } }
I20251023 02:09:45.302083 19233 catalog_manager.cc:5649] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index -1 to 29, NON_VOTER d24706c22e344a48b1f6715bd3115461 (127.15.161.197) added. New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:45.302634 19734 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
W20251023 02:09:45.306324 19700 consensus_peers.cc:597] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aece3449e9914e6a83067ce373506a58. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:45.306838 19406 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 02:09:45.307276 20265 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:45.307377 19485 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 02:09:45.307829 20265 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:45.310079 20204 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:45.310428 19406 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:45.310596 19485 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } }
I20251023 02:09:45.313457 19488 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:45.313861 19233 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 31 to 33, NON_VOTER 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) added. New cstate: current_term: 2 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:45.315898 19810 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20251023 02:09:45.316076 19406 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20251023 02:09:45.316330 20230 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
W20251023 02:09:45.317251 19454 consensus_peers.cc:597] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 -> Peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873): Couldn't send request to peer e56d3da1a4934bc59b49951222c4ba5f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 579d6f2275814c5eae528d85a86bf288. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:45.317683 20230 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:45.318429 20247 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Committing config change with OpId 5.36: config changed from index 34 to 36, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:45.318902 19810 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Committing config change with OpId 5.36: config changed from index 34 to 36, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:45.320020 19408 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Committing config change with OpId 5.36: config changed from index 34 to 36, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } } }
I20251023 02:09:45.321990 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 34 to 36, NON_VOTER e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) added. New cstate: current_term: 5 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:45.719487 20271 ts_tablet_manager.cc:933] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: Initiating tablet copy from peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:45.720238 20271 tablet_copy_client.cc:323] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:38095
I20251023 02:09:45.720715 19498 tablet_copy_service.cc:140] P 42800175d61f4a9d8a28536807976ad6: Received BeginTabletCopySession request for tablet 579d6f2275814c5eae528d85a86bf288 from peer e56d3da1a4934bc59b49951222c4ba5f ({username='slave'} at 127.0.0.1:37906)
I20251023 02:09:45.720842 19498 tablet_copy_service.cc:161] P 42800175d61f4a9d8a28536807976ad6: Beginning new tablet copy session on tablet 579d6f2275814c5eae528d85a86bf288 from peer e56d3da1a4934bc59b49951222c4ba5f at {username='slave'} at 127.0.0.1:37906: session id = e56d3da1a4934bc59b49951222c4ba5f-579d6f2275814c5eae528d85a86bf288
I20251023 02:09:45.723071 19498 tablet_copy_source_session.cc:215] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:45.723997 20271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579d6f2275814c5eae528d85a86bf288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:45.728865 20271 tablet_copy_client.cc:806] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:45.729081 20271 tablet_copy_client.cc:670] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:45.730307 20272 tablet_copy_client.cc:1025] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:45.832907 20275 ts_tablet_manager.cc:933] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: Initiating tablet copy from peer 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:45.833436 20275 tablet_copy_client.cc:323] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:36623
I20251023 02:09:45.833879 19424 tablet_copy_service.cc:140] P 6d761a2ec1dd4fed941acf04edae065f: Received BeginTabletCopySession request for tablet 38f0952ea765453aaf13876360de864f from peer e56d3da1a4934bc59b49951222c4ba5f ({username='slave'} at 127.0.0.1:35550)
I20251023 02:09:45.833997 19424 tablet_copy_service.cc:161] P 6d761a2ec1dd4fed941acf04edae065f: Beginning new tablet copy session on tablet 38f0952ea765453aaf13876360de864f from peer e56d3da1a4934bc59b49951222c4ba5f at {username='slave'} at 127.0.0.1:35550: session id = e56d3da1a4934bc59b49951222c4ba5f-38f0952ea765453aaf13876360de864f
I20251023 02:09:45.836233 19424 tablet_copy_source_session.cc:215] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:45.837440 20275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f0952ea765453aaf13876360de864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:45.838480 20277 ts_tablet_manager.cc:933] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: Initiating tablet copy from peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:45.838982 20277 tablet_copy_client.cc:323] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:43093
I20251023 02:09:45.839524 19349 tablet_copy_service.cc:140] P 5cfe6544d1b249e59e921b04cd1c5bcb: Received BeginTabletCopySession request for tablet 24ed6c24dc924a0ab03d84241f9d96ea from peer 4a65545dd26a4a018e7a9927513adaad ({username='slave'} at 127.0.0.1:40260)
I20251023 02:09:45.839660 19349 tablet_copy_service.cc:161] P 5cfe6544d1b249e59e921b04cd1c5bcb: Beginning new tablet copy session on tablet 24ed6c24dc924a0ab03d84241f9d96ea from peer 4a65545dd26a4a018e7a9927513adaad at {username='slave'} at 127.0.0.1:40260: session id = 4a65545dd26a4a018e7a9927513adaad-24ed6c24dc924a0ab03d84241f9d96ea
I20251023 02:09:45.841516 19349 tablet_copy_source_session.cc:215] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:45.842410 20277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ed6c24dc924a0ab03d84241f9d96ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:45.842973 20275 tablet_copy_client.cc:806] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:45.843217 20275 tablet_copy_client.cc:670] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:45.844724 20276 tablet_copy_client.cc:1025] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:45.847437 20277 tablet_copy_client.cc:806] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:45.847654 20277 tablet_copy_client.cc:670] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:45.848977 20278 tablet_copy_client.cc:1025] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:45.854362 20279 ts_tablet_manager.cc:933] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: Initiating tablet copy from peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093)
I20251023 02:09:45.854789 20279 tablet_copy_client.cc:323] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:43093
I20251023 02:09:45.855206 19349 tablet_copy_service.cc:140] P 5cfe6544d1b249e59e921b04cd1c5bcb: Received BeginTabletCopySession request for tablet 5dab8a517d0a41ceb8ccf4b03d137571 from peer d24706c22e344a48b1f6715bd3115461 ({username='slave'} at 127.0.0.1:40250)
I20251023 02:09:45.855283 19349 tablet_copy_service.cc:161] P 5cfe6544d1b249e59e921b04cd1c5bcb: Beginning new tablet copy session on tablet 5dab8a517d0a41ceb8ccf4b03d137571 from peer d24706c22e344a48b1f6715bd3115461 at {username='slave'} at 127.0.0.1:40250: session id = d24706c22e344a48b1f6715bd3115461-5dab8a517d0a41ceb8ccf4b03d137571
I20251023 02:09:45.856757 19349 tablet_copy_source_session.cc:215] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:45.857479 20279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab8a517d0a41ceb8ccf4b03d137571. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:45.860680 20279 tablet_copy_client.cc:806] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:45.860844 20279 tablet_copy_client.cc:670] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:45.861871 20280 tablet_copy_client.cc:1025] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:45.882349 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:45.884562 19488 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:45.884711 19488 raft_consensus.cc:606] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Received request to transfer leadership to TS d24706c22e344a48b1f6715bd3115461
I20251023 02:09:45.885473 19339 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:45.885613 19339 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:45.886410 19488 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:45.886567 19488 raft_consensus.cc:606] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:45.886886 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 02:09:45.887069 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:45.893028 20281 raft_consensus.cc:993] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:45.893138 20281 raft_consensus.cc:1081] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Signalling peer d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:45.893577 19810 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93"
dest_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:54260
I20251023 02:09:45.893711 19810 raft_consensus.cc:493] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:45.893805 19810 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:45.895838 19810 raft_consensus.cc:515] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:45.896461 19810 leader_election.cc:290] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 5 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:45.897452 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 5 candidate_status { last_received { term: 4 index: 36 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:45.897606 19488 raft_consensus.cc:3055] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20251023 02:09:45.897660 19488 raft_consensus.cc:740] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:45.897828 19488 consensus_queue.cc:260] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 4.36, Last appended by leader: 36, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:45.898046 19488 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:45.900079 19488 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 5.
I20251023 02:09:45.900519 19773 leader_election.cc:304] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, d24706c22e344a48b1f6715bd3115461; no voters:
I20251023 02:09:45.900764 20270 raft_consensus.cc:2804] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Leader election won for term 5
I20251023 02:09:45.900878 20270 raft_consensus.cc:697] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 5 LEADER]: Becoming Leader. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Running, Role: LEADER
I20251023 02:09:45.901064 20270 consensus_queue.cc:237] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 4.36, Last appended by leader: 36, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:45.902029 19733 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f28c69fe3504243ad3cb29e0c5e2d93" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 5 candidate_status { last_received { term: 4 index: 36 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:45.902184 19733 raft_consensus.cc:3060] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:45.903098 19233 catalog_manager.cc:5649] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 reported cstate change: term changed from 4 to 5, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to d24706c22e344a48b1f6715bd3115461 (127.15.161.197). New cstate: current_term: 5 leader_uuid: "d24706c22e344a48b1f6715bd3115461" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:45.904191 19733 raft_consensus.cc:2468] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 5.
I20251023 02:09:45.912591 20288 ts_tablet_manager.cc:933] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: Initiating tablet copy from peer 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:45.913041 20288 tablet_copy_client.cc:323] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:38095
I20251023 02:09:45.913456 19498 tablet_copy_service.cc:140] P 42800175d61f4a9d8a28536807976ad6: Received BeginTabletCopySession request for tablet 9973c11f67d54ed2a568e03d04d91ab2 from peer d24706c22e344a48b1f6715bd3115461 ({username='slave'} at 127.0.0.1:49684)
I20251023 02:09:45.913538 19498 tablet_copy_service.cc:161] P 42800175d61f4a9d8a28536807976ad6: Beginning new tablet copy session on tablet 9973c11f67d54ed2a568e03d04d91ab2 from peer d24706c22e344a48b1f6715bd3115461 at {username='slave'} at 127.0.0.1:49684: session id = d24706c22e344a48b1f6715bd3115461-9973c11f67d54ed2a568e03d04d91ab2
I20251023 02:09:45.915156 19498 tablet_copy_source_session.cc:215] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:45.915937 20288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9973c11f67d54ed2a568e03d04d91ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:45.919974 20288 tablet_copy_client.cc:806] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:45.920147 20288 tablet_copy_client.cc:670] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:45.921322 20289 tablet_copy_client.cc:1025] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:45.939719 20290 ts_tablet_manager.cc:933] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: Initiating tablet copy from peer e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:45.940116 20290 tablet_copy_client.cc:323] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.196:34873
I20251023 02:09:45.940565 19745 tablet_copy_service.cc:140] P e56d3da1a4934bc59b49951222c4ba5f: Received BeginTabletCopySession request for tablet aece3449e9914e6a83067ce373506a58 from peer 4a65545dd26a4a018e7a9927513adaad ({username='slave'} at 127.0.0.1:60656)
I20251023 02:09:45.940665 19745 tablet_copy_service.cc:161] P e56d3da1a4934bc59b49951222c4ba5f: Beginning new tablet copy session on tablet aece3449e9914e6a83067ce373506a58 from peer 4a65545dd26a4a018e7a9927513adaad at {username='slave'} at 127.0.0.1:60656: session id = 4a65545dd26a4a018e7a9927513adaad-aece3449e9914e6a83067ce373506a58
I20251023 02:09:45.942636 19745 tablet_copy_source_session.cc:215] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:09:45.943400 20290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aece3449e9914e6a83067ce373506a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:09:45.947299 20290 tablet_copy_client.cc:806] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 0 data blocks...
I20251023 02:09:45.947458 20290 tablet_copy_client.cc:670] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Starting download of 1 WAL segments...
I20251023 02:09:45.948563 20291 tablet_copy_client.cc:1025] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Injecting latency into file download: 2000
I20251023 02:09:46.264758 20293 raft_consensus.cc:993] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:46.265146 20293 raft_consensus.cc:1081] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:46.265614 19406 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35516
I20251023 02:09:46.265761 19406 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:46.265863 19406 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:46.268061 19406 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:46.268607 19406 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 5 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:46.269026 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 5 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:46.269176 19339 raft_consensus.cc:3055] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Stepping down as leader of term 4
I20251023 02:09:46.269243 19339 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:46.269264 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 5 candidate_status { last_received { term: 4 index: 31 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:46.269534 19488 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:46.269441 19339 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 31, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:46.269771 19339 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:46.271839 19488 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 5.
I20251023 02:09:46.271939 19339 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 5.
I20251023 02:09:46.272181 19377 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:46.272445 20274 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Leader election won for term 5
I20251023 02:09:46.272608 20274 raft_consensus.cc:697] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:46.272835 20274 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 31, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:46.274878 19233 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 4 to 5, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 5 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:46.280784 19406 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:46.280906 19406 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Received request to transfer leadership
I20251023 02:09:46.286288 19406 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:46.286424 19406 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Received request to transfer leadership
I20251023 02:09:46.287369 19406 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:46.287477 19406 raft_consensus.cc:606] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Received request to transfer leadership
W20251023 02:09:46.287811 19285 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 24ed6c24dc924a0ab03d84241f9d96ea already in progress
W20251023 02:09:46.287890 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 24ed6c24dc924a0ab03d84241f9d96ea already in progress
I20251023 02:09:46.294351 19733 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
I20251023 02:09:46.294699 20270 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:09:46.308485 20247 raft_consensus.cc:993] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:46.308626 20247 raft_consensus.cc:1081] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:46.309445 19338 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44374
I20251023 02:09:46.309638 19338 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:46.309762 19338 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:46.312492 19338 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:46.313700 19734 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:46.313854 19734 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:46.313937 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:46.314144 19485 raft_consensus.cc:3055] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20251023 02:09:46.314239 19485 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:46.314517 19485 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:46.314853 19485 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:09:46.315927 19734 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 5.
I20251023 02:09:46.316267 19305 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:46.317693 20293 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Leader election won for term 5
I20251023 02:09:46.317824 20293 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:46.317986 19485 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 5.
I20251023 02:09:46.318037 20293 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:46.313445 19338 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 5 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:46.320293 19488 raft_consensus.cc:1275] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 4 index: 36. Preceding OpId from leader: term: 5 index: 37. (index mismatch)
I20251023 02:09:46.320770 20298 consensus_queue.cc:1048] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:09:46.320708 19233 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 4 to 5, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 5 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:46.698791 19338 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 5 index: 32. (index mismatch)
I20251023 02:09:46.699221 20257 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:46.704566 19488 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 5 index: 32. (index mismatch)
I20251023 02:09:46.705044 20257 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:46.708468 20306 raft_consensus.cc:993] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:46.708603 20306 raft_consensus.cc:1081] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:46.709151 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:09:46.709350 19485 raft_consensus.cc:493] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:46.709431 19485 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Advancing to term 6
W20251023 02:09:46.711205 19379 consensus_peers.cc:597] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:46.714087 19485 raft_consensus.cc:515] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:46.715174 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 32 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:46.716081 19485 leader_election.cc:290] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623)
I20251023 02:09:46.716928 19408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "24ed6c24dc924a0ab03d84241f9d96ea" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 32 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:46.717093 19408 raft_consensus.cc:3055] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Stepping down as leader of term 5
I20251023 02:09:46.717150 19408 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:46.717350 19408 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 5.32, Last appended by leader: 32, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:46.717571 19408 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:46.719772 19408 raft_consensus.cc:2468] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 6.
I20251023 02:09:46.720085 19451 leader_election.cc:304] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters: 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:46.720314 20247 raft_consensus.cc:2804] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20251023 02:09:46.720422 20247 raft_consensus.cc:697] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:46.720602 20247 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 5.32, Last appended by leader: 32, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:46.722568 19232 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 5 to 6, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 6 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:46.731079 19733 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 02:09:46.731417 20219 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:46.733870 19488 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 02:09:46.735435 20219 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
W20251023 02:09:47.148285 19453 consensus_peers.cc:597] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 -> Peer 4a65545dd26a4a018e7a9927513adaad (127.15.161.198:38041): Couldn't send request to peer 4a65545dd26a4a018e7a9927513adaad. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:09:47.190165 19338 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:47.192452 19338 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 32. Preceding OpId from leader: term: 6 index: 33. (index mismatch)
I20251023 02:09:47.192909 20292 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:47.195261 19406 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 32. Preceding OpId from leader: term: 6 index: 33. (index mismatch)
I20251023 02:09:47.195941 20281 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:47.730610 20271 tablet_copy_client.cc:538] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:47.734061 20271 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap starting.
I20251023 02:09:47.777676 20271 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=36 overwritten=0 applied=36 ignored=0} inserts{seen=109 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:47.777981 20271 tablet_bootstrap.cc:492] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap complete.
I20251023 02:09:47.778229 20271 ts_tablet_manager.cc:1403] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: Time spent bootstrapping tablet: real 0.044s user 0.037s sys 0.008s
I20251023 02:09:47.778816 20271 raft_consensus.cc:359] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:47.778960 20271 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Initialized, Role: LEARNER
I20251023 02:09:47.779124 20271 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:47.779596 20271 ts_tablet_manager.cc:1434] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:09:47.780136 19498 tablet_copy_service.cc:342] P 42800175d61f4a9d8a28536807976ad6: Request end of tablet copy session e56d3da1a4934bc59b49951222c4ba5f-579d6f2275814c5eae528d85a86bf288 received from {username='slave'} at 127.0.0.1:37906
I20251023 02:09:47.780246 19498 tablet_copy_service.cc:434] P 42800175d61f4a9d8a28536807976ad6: ending tablet copy session e56d3da1a4934bc59b49951222c4ba5f-579d6f2275814c5eae528d85a86bf288 on tablet 579d6f2275814c5eae528d85a86bf288 with peer e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:47.845016 20275 tablet_copy_client.cc:538] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:47.847714 20275 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap starting.
I20251023 02:09:47.849321 20277 tablet_copy_client.cc:538] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:47.851801 20277 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: Bootstrap starting.
I20251023 02:09:47.862167 20279 tablet_copy_client.cc:538] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:47.864650 20279 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: Bootstrap starting.
I20251023 02:09:47.884421 20275 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=121 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:47.884807 20275 tablet_bootstrap.cc:492] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: Bootstrap complete.
I20251023 02:09:47.885048 20275 ts_tablet_manager.cc:1403] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: Time spent bootstrapping tablet: real 0.037s user 0.038s sys 0.000s
I20251023 02:09:47.885608 20275 raft_consensus.cc:359] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:47.885852 20275 raft_consensus.cc:740] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d3da1a4934bc59b49951222c4ba5f, State: Initialized, Role: LEARNER
I20251023 02:09:47.886072 20275 consensus_queue.cc:260] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:47.887389 20275 ts_tablet_manager.cc:1434] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:09:47.887552 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:47.888000 19424 tablet_copy_service.cc:342] P 6d761a2ec1dd4fed941acf04edae065f: Request end of tablet copy session e56d3da1a4934bc59b49951222c4ba5f-38f0952ea765453aaf13876360de864f received from {username='slave'} at 127.0.0.1:35550
I20251023 02:09:47.888093 19424 tablet_copy_service.cc:434] P 6d761a2ec1dd4fed941acf04edae065f: ending tablet copy session e56d3da1a4934bc59b49951222c4ba5f-38f0952ea765453aaf13876360de864f on tablet 38f0952ea765453aaf13876360de864f with peer e56d3da1a4934bc59b49951222c4ba5f
I20251023 02:09:47.892234 19488 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:47.892424 19488 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:47.893329 19338 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "4a65545dd26a4a018e7a9927513adaad"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:47.893461 19338 raft_consensus.cc:606] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Received request to transfer leadership to TS 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:47.894625 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251023 02:09:47.894865 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:47.900184 20277 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=124 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:47.900524 20277 tablet_bootstrap.cc:492] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: Bootstrap complete.
I20251023 02:09:47.900732 20277 ts_tablet_manager.cc:1403] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: Time spent bootstrapping tablet: real 0.049s user 0.041s sys 0.000s
I20251023 02:09:47.901430 20277 raft_consensus.cc:359] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:47.901607 20277 raft_consensus.cc:740] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Initialized, Role: LEARNER
I20251023 02:09:47.901813 20277 consensus_queue.cc:260] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 4.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:47.902524 20277 ts_tablet_manager.cc:1434] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20251023 02:09:47.903092 19349 tablet_copy_service.cc:342] P 5cfe6544d1b249e59e921b04cd1c5bcb: Request end of tablet copy session 4a65545dd26a4a018e7a9927513adaad-24ed6c24dc924a0ab03d84241f9d96ea received from {username='slave'} at 127.0.0.1:40260
I20251023 02:09:47.903177 19349 tablet_copy_service.cc:434] P 5cfe6544d1b249e59e921b04cd1c5bcb: ending tablet copy session 4a65545dd26a4a018e7a9927513adaad-24ed6c24dc924a0ab03d84241f9d96ea on tablet 24ed6c24dc924a0ab03d84241f9d96ea with peer 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:47.921595 20288 tablet_copy_client.cc:538] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:47.921623 20279 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:47.921942 20279 tablet_bootstrap.cc:492] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: Bootstrap complete.
I20251023 02:09:47.922164 20279 ts_tablet_manager.cc:1403] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: Time spent bootstrapping tablet: real 0.058s user 0.042s sys 0.014s
I20251023 02:09:47.922720 20279 raft_consensus.cc:359] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.922889 20279 raft_consensus.cc:740] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Initialized, Role: LEARNER
I20251023 02:09:47.923049 20279 consensus_queue.cc:260] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.923588 20279 ts_tablet_manager.cc:1434] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:09:47.924089 19349 tablet_copy_service.cc:342] P 5cfe6544d1b249e59e921b04cd1c5bcb: Request end of tablet copy session d24706c22e344a48b1f6715bd3115461-5dab8a517d0a41ceb8ccf4b03d137571 received from {username='slave'} at 127.0.0.1:40250
I20251023 02:09:47.924204 19349 tablet_copy_service.cc:434] P 5cfe6544d1b249e59e921b04cd1c5bcb: ending tablet copy session d24706c22e344a48b1f6715bd3115461-5dab8a517d0a41ceb8ccf4b03d137571 on tablet 5dab8a517d0a41ceb8ccf4b03d137571 with peer d24706c22e344a48b1f6715bd3115461
I20251023 02:09:47.924474 20288 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: Bootstrap starting.
I20251023 02:09:47.948864 20290 tablet_copy_client.cc:538] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:09:47.951231 20290 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap starting.
I20251023 02:09:47.958022 20324 raft_consensus.cc:993] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 4a65545dd26a4a018e7a9927513adaad to start an election
I20251023 02:09:47.958209 20293 raft_consensus.cc:1081] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Signalling peer 4a65545dd26a4a018e7a9927513adaad to start an election
I20251023 02:09:47.958619 19893 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571"
dest_uuid: "4a65545dd26a4a018e7a9927513adaad"
from {username='slave'} at 127.0.0.1:54930
I20251023 02:09:47.958758 19893 raft_consensus.cc:493] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:47.958837 19893 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:47.959724 20288 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=107 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:47.960021 20288 tablet_bootstrap.cc:492] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: Bootstrap complete.
I20251023 02:09:47.960213 20288 ts_tablet_manager.cc:1403] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: Time spent bootstrapping tablet: real 0.036s user 0.027s sys 0.008s
I20251023 02:09:47.960669 20288 raft_consensus.cc:359] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.960796 20288 raft_consensus.cc:740] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Initialized, Role: LEARNER
I20251023 02:09:47.960973 20288 consensus_queue.cc:260] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.961264 19893 raft_consensus.cc:515] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.961453 20288 ts_tablet_manager.cc:1434] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20251023 02:09:47.961968 19498 tablet_copy_service.cc:342] P 42800175d61f4a9d8a28536807976ad6: Request end of tablet copy session d24706c22e344a48b1f6715bd3115461-9973c11f67d54ed2a568e03d04d91ab2 received from {username='slave'} at 127.0.0.1:49684
I20251023 02:09:47.962007 19893 leader_election.cc:290] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [CANDIDATE]: Term 2 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095)
I20251023 02:09:47.962054 19498 tablet_copy_service.cc:434] P 42800175d61f4a9d8a28536807976ad6: ending tablet copy session d24706c22e344a48b1f6715bd3115461-9973c11f67d54ed2a568e03d04d91ab2 on tablet 9973c11f67d54ed2a568e03d04d91ab2 with peer d24706c22e344a48b1f6715bd3115461
I20251023 02:09:47.962265 19338 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571" candidate_uuid: "4a65545dd26a4a018e7a9927513adaad" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:47.962441 19338 raft_consensus.cc:3055] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:09:47.962533 19338 raft_consensus.cc:740] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:47.962779 19338 consensus_queue.cc:260] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.963187 19338 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:47.965404 19338 raft_consensus.cc:2468] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a65545dd26a4a018e7a9927513adaad in term 2.
I20251023 02:09:47.966096 19850 leader_election.cc:304] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a65545dd26a4a018e7a9927513adaad, 5cfe6544d1b249e59e921b04cd1c5bcb; no voters:
I20251023 02:09:47.966418 20322 raft_consensus.cc:2804] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:09:47.972810 20322 raft_consensus.cc:697] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEADER]: Becoming Leader. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Running, Role: LEADER
I20251023 02:09:47.973157 20322 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } }
I20251023 02:09:47.980356 19232 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad reported cstate change: term changed from 1 to 2, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 4a65545dd26a4a018e7a9927513adaad (127.15.161.198). New cstate: current_term: 2 leader_uuid: "4a65545dd26a4a018e7a9927513adaad" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:47.981061 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5dab8a517d0a41ceb8ccf4b03d137571" candidate_uuid: "4a65545dd26a4a018e7a9927513adaad" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:47.981211 19488 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:09:47.983345 19488 raft_consensus.cc:2468] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a65545dd26a4a018e7a9927513adaad in term 2.
I20251023 02:09:48.000708 20290 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=108 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:09:48.001058 20290 tablet_bootstrap.cc:492] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: Bootstrap complete.
I20251023 02:09:48.001255 20290 ts_tablet_manager.cc:1403] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: Time spent bootstrapping tablet: real 0.050s user 0.038s sys 0.004s
I20251023 02:09:48.001755 20290 raft_consensus.cc:359] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:48.001901 20290 raft_consensus.cc:740] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4a65545dd26a4a018e7a9927513adaad, State: Initialized, Role: LEARNER
I20251023 02:09:48.002051 20290 consensus_queue.cc:260] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } }
I20251023 02:09:48.002460 20290 ts_tablet_manager.cc:1434] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20251023 02:09:48.002967 19745 tablet_copy_service.cc:342] P e56d3da1a4934bc59b49951222c4ba5f: Request end of tablet copy session 4a65545dd26a4a018e7a9927513adaad-aece3449e9914e6a83067ce373506a58 received from {username='slave'} at 127.0.0.1:60656
I20251023 02:09:48.003077 19745 tablet_copy_service.cc:434] P e56d3da1a4934bc59b49951222c4ba5f: ending tablet copy session 4a65545dd26a4a018e7a9927513adaad-aece3449e9914e6a83067ce373506a58 on tablet aece3449e9914e6a83067ce373506a58 with peer 4a65545dd26a4a018e7a9927513adaad
I20251023 02:09:48.005961 19733 raft_consensus.cc:1217] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.31->[1.32-1.32] Dedup: 1.32->[]
I20251023 02:09:48.169066 19893 raft_consensus.cc:3060] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 4 LEARNER]: Advancing to term 6
I20251023 02:09:48.170961 19893 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 5 index: 32. (index mismatch)
I20251023 02:09:48.171451 20317 consensus_queue.cc:1050] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:48.188586 20317 raft_consensus.cc:1064] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: attempting to promote NON_VOTER 4a65545dd26a4a018e7a9927513adaad to VOTER
I20251023 02:09:48.189260 20317 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 6.33, Last appended by leader: 32, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:48.191241 19406 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251023 02:09:48.191732 19338 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251023 02:09:48.191844 20317 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.191958 19894 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251023 02:09:48.192176 20316 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.192339 20317 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.194322 20247 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Committing config change with OpId 6.34: config changed from index 31 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.197054 19338 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Committing config change with OpId 6.34: config changed from index 31 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.197810 19232 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 31 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:48.197052 19893 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 FOLLOWER]: Committing config change with OpId 6.34: config changed from index 31 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.200989 19488 consensus_queue.cc:237] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 6.34, Last appended by leader: 32, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:48.203075 19406 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Committing config change with OpId 6.34: config changed from index 31 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.203652 19893 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251023 02:09:48.204236 20317 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:48.206488 19339 raft_consensus.cc:1275] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251023 02:09:48.207306 20317 consensus_queue.cc:1048] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:48.208343 20247 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Committing config change with OpId 6.35: config changed from index 34 to 35, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.211025 19893 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 FOLLOWER]: Committing config change with OpId 6.35: config changed from index 34 to 35, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.211024 19338 raft_consensus.cc:2955] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Committing config change with OpId 6.35: config changed from index 34 to 35, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.210683 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 24ed6c24dc924a0ab03d84241f9d96ea with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:48.215227 19232 catalog_manager.cc:5649] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: config changed from index 34 to 35, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New cstate: current_term: 6 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.218710 19402 tablet_service.cc:1552] Processing DeleteTablet for tablet 24ed6c24dc924a0ab03d84241f9d96ea with delete_type TABLET_DATA_TOMBSTONED (TS 6d761a2ec1dd4fed941acf04edae065f not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:48.219252 20340 tablet_replica.cc:333] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:09:48.219429 20340 raft_consensus.cc:2243] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:48.219676 20340 raft_consensus.cc:2272] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:48.220317 20340 ts_tablet_manager.cc:1916] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:48.225991 20340 ts_tablet_manager.cc:1929] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.34
I20251023 02:09:48.226157 20340 log.cc:1199] T 24ed6c24dc924a0ab03d84241f9d96ea P 6d761a2ec1dd4fed941acf04edae065f: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/24ed6c24dc924a0ab03d84241f9d96ea
I20251023 02:09:48.226833 19221 catalog_manager.cc:4985] TS 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): tablet 24ed6c24dc924a0ab03d84241f9d96ea (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:48.273389 20338 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:48.273523 20338 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:09:48.273967 19406 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:09:48.274108 19406 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:48.274232 19406 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:48.275804 19406 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:48.276310 19406 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 6 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:09:48.276669 19488 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:48.276818 19488 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20251023 02:09:48.276916 19488 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:48.277153 19488 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:48.277516 19488 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:48.279531 19488 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 6.
I20251023 02:09:48.279763 19810 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:09:48.279845 19377 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:48.279934 19810 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:48.282012 19810 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 6.
I20251023 02:09:48.282338 20257 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Leader election won for term 6
I20251023 02:09:48.282725 20257 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:48.283015 20257 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:48.285647 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 5 to 6, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 6 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:48.292873 19406 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:48.293007 19406 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Received request to transfer leadership
I20251023 02:09:48.297812 19406 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:48.297946 19406 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Received request to transfer leadership
W20251023 02:09:48.298247 19221 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 579d6f2275814c5eae528d85a86bf288 already in progress [suppressed 1 similar messages]
I20251023 02:09:48.299046 19406 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:48.299189 19406 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Received request to transfer leadership
W20251023 02:09:48.299548 19285 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 579d6f2275814c5eae528d85a86bf288 already in progress
W20251023 02:09:48.299649 19285 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 579d6f2275814c5eae528d85a86bf288 already in progress
I20251023 02:09:48.351675 19893 raft_consensus.cc:1217] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEARNER]: Deduplicated request from leader. Original: 2.32->[2.33-2.33] Dedup: 2.33->[]
I20251023 02:09:48.419096 19810 raft_consensus.cc:3060] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Advancing to term 2
I20251023 02:09:48.421324 19810 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 LEARNER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:48.421729 20328 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: NON_VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:48.423527 19809 raft_consensus.cc:1217] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 02:09:48.424412 20322 raft_consensus.cc:1064] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
W20251023 02:09:48.424537 20322 raft_consensus.cc:1068] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: Unable to promote non-voter d24706c22e344a48b1f6715bd3115461: Illegal state: Leader has not yet committed an operation in its own term
I20251023 02:09:48.431530 19338 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:48.431936 20328 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:48.435149 19485 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:09:48.435559 20328 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:09:48.447409 20347 raft_consensus.cc:1064] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
I20251023 02:09:48.448004 20347 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 31, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:48.450074 19339 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 02:09:48.450282 19810 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 LEARNER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 02:09:48.450839 20322 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:48.451318 20347 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:48.454041 19488 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 02:09:48.453893 20322 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.454432 19810 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.455364 20328 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:48.457868 19485 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.458683 19339 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.459167 19230 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 reported cstate change: config changed from index 31 to 33, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4a65545dd26a4a018e7a9927513adaad" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.463264 19893 consensus_queue.cc:237] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:48.464475 19810 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 02:09:48.464706 19339 raft_consensus.cc:1275] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Refusing update from remote peer 4a65545dd26a4a018e7a9927513adaad: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 02:09:48.465077 20322 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.465281 20347 consensus_queue.cc:1048] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.466835 20347 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.467725 19809 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.470348 19283 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 5dab8a517d0a41ceb8ccf4b03d137571 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:48.470779 19339 raft_consensus.cc:2955] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.470914 19232 catalog_manager.cc:5649] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad reported cstate change: config changed from index 33 to 34, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New cstate: current_term: 2 leader_uuid: "4a65545dd26a4a018e7a9927513adaad" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:48.473789 19473 tablet_service.cc:1552] Processing DeleteTablet for tablet 5dab8a517d0a41ceb8ccf4b03d137571 with delete_type TABLET_DATA_TOMBSTONED (TS 42800175d61f4a9d8a28536807976ad6 not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:37842
I20251023 02:09:48.474538 20352 tablet_replica.cc:333] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:09:48.474742 20352 raft_consensus.cc:2243] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:48.475068 20352 raft_consensus.cc:2272] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:48.476198 20352 ts_tablet_manager.cc:1916] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:48.479967 20352 ts_tablet_manager.cc:1929] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.33
I20251023 02:09:48.480050 20352 log.cc:1199] T 5dab8a517d0a41ceb8ccf4b03d137571 P 42800175d61f4a9d8a28536807976ad6: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/5dab8a517d0a41ceb8ccf4b03d137571
I20251023 02:09:48.480581 19221 catalog_manager.cc:4985] TS 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): tablet 5dab8a517d0a41ceb8ccf4b03d137571 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:48.516371 20341 raft_consensus.cc:1064] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
I20251023 02:09:48.516968 20341 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:48.518704 19893 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 02:09:48.518787 19339 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 02:09:48.519176 19733 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 02:09:48.519147 20353 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:48.519565 20257 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:48.519732 20353 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:09:48.523160 20257 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index 32 to 33, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.523607 19893 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.525836 19339 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.527454 19230 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: config changed from index 32 to 33, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:48.528726 19733 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.534014 19406 consensus_queue.cc:237] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:48.535171 19733 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 1 index: 34. (index mismatch)
I20251023 02:09:48.535185 19893 raft_consensus.cc:1275] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 1 index: 34. (index mismatch)
I20251023 02:09:48.535519 20341 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.535702 20353 consensus_queue.cc:1048] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.537602 20257 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Committing config change with OpId 1.34: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.538025 19893 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Committing config change with OpId 1.34: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.538187 19734 raft_consensus.cc:2955] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Committing config change with OpId 1.34: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.540328 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 38f0952ea765453aaf13876360de864f with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:48.541426 19232 catalog_manager.cc:5649] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad reported cstate change: config changed from index 33 to 34, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:48.544054 19326 tablet_service.cc:1552] Processing DeleteTablet for tablet 38f0952ea765453aaf13876360de864f with delete_type TABLET_DATA_TOMBSTONED (TS 5cfe6544d1b249e59e921b04cd1c5bcb not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:48.544565 20359 tablet_replica.cc:333] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:09:48.544724 20359 raft_consensus.cc:2243] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:48.544900 20359 raft_consensus.cc:2272] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:48.545584 20359 ts_tablet_manager.cc:1916] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:48.549485 20359 ts_tablet_manager.cc:1929] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.33
I20251023 02:09:48.549607 20359 log.cc:1199] T 38f0952ea765453aaf13876360de864f P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/38f0952ea765453aaf13876360de864f
I20251023 02:09:48.550146 19222 catalog_manager.cc:4985] TS 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): tablet 38f0952ea765453aaf13876360de864f (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:48.750931 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251023 02:09:48.751327 20341 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:09:48.753167 20355 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:48.753279 20257 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:48.753988 19487 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:09:48.754179 19487 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:48.754273 19487 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Advancing to term 7
I20251023 02:09:48.757583 19487 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:48.758543 19408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:48.758657 19485 raft_consensus.cc:1240] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6d761a2ec1dd4fed941acf04edae065f for earlier term 6. Current term is 7. Ops: []
I20251023 02:09:48.759366 19408 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Stepping down as leader of term 6
I20251023 02:09:48.759433 19408 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:09:48.759429 19734 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 5 LEARNER]: Advancing to term 6
I20251023 02:09:48.759644 19408 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:48.759742 19809 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 6 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251023 02:09:48.759820 19408 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 6 FOLLOWER]: Advancing to term 7
I20251023 02:09:48.760541 19809 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:09:48.760653 19809 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 6 FOLLOWER]: Advancing to term 7
I20251023 02:09:48.761412 19734 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 6 LEARNER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251023 02:09:48.761941 19408 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 7.
I20251023 02:09:48.762423 19487 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:09:48.762440 19451 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:48.762782 19809 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 7.
I20251023 02:09:48.762797 20338 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20251023 02:09:48.762990 20338 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:48.763218 20338 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 6.37, Last appended by leader: 37, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } }
I20251023 02:09:48.765211 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 6 to 7, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 7 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:48.856511 20363 raft_consensus.cc:1064] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: attempting to promote NON_VOTER 4a65545dd26a4a018e7a9927513adaad to VOTER
I20251023 02:09:48.857682 20363 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 31, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:48.858325 20364 raft_consensus.cc:1064] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: attempting to promote NON_VOTER d24706c22e344a48b1f6715bd3115461 to VOTER
I20251023 02:09:48.858968 20364 consensus_queue.cc:237] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:48.859894 19893 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEARNER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 02:09:48.860312 20304 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.860828 19485 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 02:09:48.861128 19408 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 02:09:48.861649 19809 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:48.862136 20363 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.862440 19408 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:48.862408 20304 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 02:09:48.862593 20364 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:48.862213 19339 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 02:09:48.862951 20338 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:48.863336 20338 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 02:09:48.868263 20338 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.872596 19339 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.874725 20362 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.875576 19408 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.876890 19230 catalog_manager.cc:5649] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 29 to 30, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:48.877897 19894 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.882000 19406 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.883654 19809 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, d24706c22e344a48b1f6715bd3115461 (127.15.161.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } attrs { replace: true } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.886754 19230 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad reported cstate change: config changed from index 33 to 34, 4a65545dd26a4a018e7a9927513adaad (127.15.161.198) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.888393 19487 consensus_queue.cc:237] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:48.891938 19408 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:48.892422 20364 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:48.892596 19809 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 02:09:48.893713 19733 consensus_queue.cc:237] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } }
I20251023 02:09:48.894447 20338 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:48.895151 19894 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251023 02:09:48.895042 20338 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.895807 20362 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:48.895628 19406 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.896178 19408 raft_consensus.cc:1275] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Refusing update from remote peer e56d3da1a4934bc59b49951222c4ba5f: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251023 02:09:48.897583 20304 consensus_queue.cc:1048] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:09:48.898908 19809 raft_consensus.cc:2955] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.899719 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 9973c11f67d54ed2a568e03d04d91ab2 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:48.900019 20304 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.900661 19408 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.902577 19233 catalog_manager.cc:5649] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 reported cstate change: config changed from index 30 to 31, VOTER 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } }
I20251023 02:09:48.903750 19284 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet aece3449e9914e6a83067ce373506a58 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:48.905012 19230 catalog_manager.cc:5649] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 34 to 35, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New cstate: current_term: 2 leader_uuid: "e56d3da1a4934bc59b49951222c4ba5f" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:48.905177 19326 tablet_service.cc:1552] Processing DeleteTablet for tablet 9973c11f67d54ed2a568e03d04d91ab2 with delete_type TABLET_DATA_TOMBSTONED (TS 5cfe6544d1b249e59e921b04cd1c5bcb not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:44372
I20251023 02:09:48.905470 20359 tablet_replica.cc:333] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:09:48.905648 20359 raft_consensus.cc:2243] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:48.905833 20359 raft_consensus.cc:2272] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:48.907370 20359 ts_tablet_manager.cc:1916] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:48.908485 19473 tablet_service.cc:1552] Processing DeleteTablet for tablet aece3449e9914e6a83067ce373506a58 with delete_type TABLET_DATA_TOMBSTONED (TS 42800175d61f4a9d8a28536807976ad6 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:37842
I20251023 02:09:48.908800 20352 tablet_replica.cc:333] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:09:48.908963 20352 raft_consensus.cc:2243] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:48.909086 20352 pending_rounds.cc:70] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Trying to abort 1 pending ops.
I20251023 02:09:48.909147 20352 pending_rounds.cc:77] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Aborting op as it isn't in flight: id { term: 2 index: 34 } timestamp: 7213815352757637120 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "aece3449e9914e6a83067ce373506a58" old_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: NON_VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: true } } } new_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } attrs { replace: true } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } } }
I20251023 02:09:48.909374 20352 raft_consensus.cc:2917] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Aborting config change with OpId 2.34: Aborted: Op aborted
I20251023 02:09:48.909495 20352 raft_consensus.cc:2889] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20251023 02:09:48.909577 20352 raft_consensus.cc:2272] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:48.910243 20352 ts_tablet_manager.cc:1916] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:48.912256 20359 ts_tablet_manager.cc:1929] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251023 02:09:48.912359 20359 log.cc:1199] T 9973c11f67d54ed2a568e03d04d91ab2 P 5cfe6544d1b249e59e921b04cd1c5bcb: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/9973c11f67d54ed2a568e03d04d91ab2
I20251023 02:09:48.912899 19222 catalog_manager.cc:4985] TS 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): tablet 9973c11f67d54ed2a568e03d04d91ab2 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:48.913596 19894 raft_consensus.cc:2955] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } peers { permanent_uuid: "4a65545dd26a4a018e7a9927513adaad" member_type: VOTER last_known_addr { host: "127.15.161.198" port: 38041 } attrs { promote: false } } }
I20251023 02:09:48.915212 20352 ts_tablet_manager.cc:1929] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.34
I20251023 02:09:48.915318 20352 log.cc:1199] T aece3449e9914e6a83067ce373506a58 P 42800175d61f4a9d8a28536807976ad6: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/aece3449e9914e6a83067ce373506a58
I20251023 02:09:48.915871 19221 catalog_manager.cc:4985] TS 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095): tablet aece3449e9914e6a83067ce373506a58 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:49.188163 19408 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 7 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 37. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20251023 02:09:49.188553 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251023 02:09:49.191537 19734 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 6 LEARNER]: Advancing to term 7
I20251023 02:09:49.192128 19809 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20251023 02:09:49.192740 20338 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 36, Time since last communication: 0.001s
I20251023 02:09:49.194093 19734 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 7 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20251023 02:09:49.195611 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:09:49.198828 20338 raft_consensus.cc:1064] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: attempting to promote NON_VOTER e56d3da1a4934bc59b49951222c4ba5f to VOTER
I20251023 02:09:49.199362 20338 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 7.38, Last appended by leader: 37, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:49.201208 19408 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 7 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 7 index: 39. (index mismatch)
I20251023 02:09:49.201259 19809 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 7 index: 39. (index mismatch)
I20251023 02:09:49.201618 19734 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 7 LEARNER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 7 index: 39. (index mismatch)
I20251023 02:09:49.201766 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251023 02:09:49.201967 20338 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251023 02:09:49.202371 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251023 02:09:49.204329 20364 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Committing config change with OpId 7.39: config changed from index 36 to 39, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.204946 19734 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Committing config change with OpId 7.39: config changed from index 36 to 39, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.205090 19809 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Committing config change with OpId 7.39: config changed from index 36 to 39, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.208617 19408 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 7 FOLLOWER]: Committing config change with OpId 7.39: config changed from index 36 to 39, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.208875 19232 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f reported cstate change: config changed from index 36 to 39, e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } attrs { replace: true } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.213663 19485 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 7.39, Last appended by leader: 37, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:49.214921 19734 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 39. Preceding OpId from leader: term: 7 index: 40. (index mismatch)
I20251023 02:09:49.215080 19809 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 39. Preceding OpId from leader: term: 7 index: 40. (index mismatch)
I20251023 02:09:49.215360 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20251023 02:09:49.215556 20338 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20251023 02:09:49.217465 20339 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Committing config change with OpId 7.40: config changed from index 39 to 40, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.218031 19809 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Committing config change with OpId 7.40: config changed from index 39 to 40, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.218355 19734 raft_consensus.cc:2955] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Committing config change with OpId 7.40: config changed from index 39 to 40, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } } }
I20251023 02:09:49.220459 19221 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 579d6f2275814c5eae528d85a86bf288 with cas_config_opid_index 39: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:09:49.221678 19230 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: config changed from index 39 to 40, VOTER 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) evicted. New cstate: current_term: 7 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:09:49.224143 19402 tablet_service.cc:1552] Processing DeleteTablet for tablet 579d6f2275814c5eae528d85a86bf288 with delete_type TABLET_DATA_TOMBSTONED (TS 6d761a2ec1dd4fed941acf04edae065f not found in new config with opid_index 40) from {username='slave'} at 127.0.0.1:35490
I20251023 02:09:49.224756 20379 tablet_replica.cc:333] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:09:49.224920 20379 raft_consensus.cc:2243] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 7 FOLLOWER]: Raft consensus shutting down.
I20251023 02:09:49.225137 20379 raft_consensus.cc:2272] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 02:09:49.225733 20379 ts_tablet_manager.cc:1916] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:09:49.229290 20379 ts_tablet_manager.cc:1929] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.39
I20251023 02:09:49.229375 20379 log.cc:1199] T 579d6f2275814c5eae528d85a86bf288 P 6d761a2ec1dd4fed941acf04edae065f: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/579d6f2275814c5eae528d85a86bf288
I20251023 02:09:49.229818 19221 catalog_manager.cc:4985] TS 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623): tablet 579d6f2275814c5eae528d85a86bf288 (table test-workload [id=dd092623f4f24c89ab5431dbb16a1b0a]) successfully deleted
I20251023 02:09:49.895280 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:49.897776 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:49.897925 19485 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Received request to transfer leadership to TS d24706c22e344a48b1f6715bd3115461
I20251023 02:09:49.898336 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:49.898599 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:50.348217 20364 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:50.348354 20364 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Signalling peer d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:50.348801 19809 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:54260
I20251023 02:09:50.348940 19809 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:50.349021 19809 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 7 FOLLOWER]: Advancing to term 8
I20251023 02:09:50.350643 19809 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:50.351063 19809 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 8 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:50.351377 19734 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 8 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:50.351377 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 8 candidate_status { last_received { term: 7 index: 40 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:50.351536 19734 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Advancing to term 8
I20251023 02:09:50.351598 19485 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20251023 02:09:50.351660 19485 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:50.351861 19485 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 7.40, Last appended by leader: 40, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:50.352099 19485 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Advancing to term 8
I20251023 02:09:50.353055 19734 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 8.
I20251023 02:09:50.353413 19776 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d24706c22e344a48b1f6715bd3115461, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:50.353642 20298 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 FOLLOWER]: Leader election won for term 8
I20251023 02:09:50.353705 19485 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 8.
I20251023 02:09:50.353957 20298 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 LEADER]: Becoming Leader. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Running, Role: LEADER
I20251023 02:09:50.354372 20298 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 7.40, Last appended by leader: 40, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:50.356135 19232 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 reported cstate change: term changed from 7 to 8, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to d24706c22e344a48b1f6715bd3115461 (127.15.161.197). New cstate: current_term: 8 leader_uuid: "d24706c22e344a48b1f6715bd3115461" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:50.779702 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 8 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 7 index: 40. Preceding OpId from leader: term: 8 index: 41. (index mismatch)
I20251023 02:09:50.780097 20298 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20251023 02:09:50.783457 19734 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 8 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 7 index: 40. Preceding OpId from leader: term: 8 index: 41. (index mismatch)
I20251023 02:09:50.784121 20298 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20251023 02:09:51.898888 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:51.907684 19810 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "d24706c22e344a48b1f6715bd3115461"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:43248
I20251023 02:09:51.907827 19810 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:51.908156 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:51.908303 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:51.912158 20397 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:51.912268 20397 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:51.912781 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:49684
I20251023 02:09:51.912902 19485 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:51.912976 19485 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 8 FOLLOWER]: Advancing to term 9
I20251023 02:09:51.914649 19485 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:51.915107 19485 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 9 election: Requested vote from peers d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:51.915428 19810 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 41 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:09:51.915535 19810 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 LEADER]: Stepping down as leader of term 8
I20251023 02:09:51.915495 19733 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 41 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:51.915601 19810 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Running, Role: LEADER
I20251023 02:09:51.915655 19733 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 8 FOLLOWER]: Advancing to term 9
I20251023 02:09:51.915802 19810 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 8.41, Last appended by leader: 41, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:51.916031 19810 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 8 FOLLOWER]: Advancing to term 9
I20251023 02:09:51.917336 19733 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 9.
I20251023 02:09:51.917672 19810 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 9.
I20251023 02:09:51.917724 19454 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:51.917960 20364 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20251023 02:09:51.918255 20364 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:51.918445 20364 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 8.41, Last appended by leader: 41, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:51.920186 19233 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 8 to 9, leader changed from d24706c22e344a48b1f6715bd3115461 (127.15.161.197) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 9 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:52.329456 19733 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 9 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 41. Preceding OpId from leader: term: 9 index: 42. (index mismatch)
I20251023 02:09:52.329890 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20251023 02:09:52.332866 19810 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 9 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 41. Preceding OpId from leader: term: 9 index: 42. (index mismatch)
I20251023 02:09:52.333297 20364 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20251023 02:09:53.908584 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:53.910768 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:53.910928 19485 raft_consensus.cc:606] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 LEADER]: Received request to transfer leadership to TS d24706c22e344a48b1f6715bd3115461
I20251023 02:09:53.911303 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:53.911487 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:54.137311 20411 raft_consensus.cc:993] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:54.137566 20411 raft_consensus.cc:1081] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 LEADER]: Signalling peer d24706c22e344a48b1f6715bd3115461 to start an election
I20251023 02:09:54.138325 19810 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288"
dest_uuid: "d24706c22e344a48b1f6715bd3115461"
from {username='slave'} at 127.0.0.1:54260
I20251023 02:09:54.138478 19810 raft_consensus.cc:493] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:54.138556 19810 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 9 FOLLOWER]: Advancing to term 10
I20251023 02:09:54.140177 19810 raft_consensus.cc:515] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:54.140606 19810 leader_election.cc:290] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 10 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:54.140954 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:09:54.140919 19733 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "579d6f2275814c5eae528d85a86bf288" candidate_uuid: "d24706c22e344a48b1f6715bd3115461" candidate_term: 10 candidate_status { last_received { term: 9 index: 42 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:54.141109 19485 raft_consensus.cc:3055] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20251023 02:09:54.141109 19733 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 9 FOLLOWER]: Advancing to term 10
I20251023 02:09:54.141240 19485 raft_consensus.cc:740] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:54.141486 19485 consensus_queue.cc:260] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 9.42, Last appended by leader: 42, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:54.141801 19485 raft_consensus.cc:3060] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 9 FOLLOWER]: Advancing to term 10
I20251023 02:09:54.143462 19733 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 10.
I20251023 02:09:54.143782 19776 leader_election.cc:304] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d24706c22e344a48b1f6715bd3115461, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:54.144070 20397 raft_consensus.cc:2804] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 10 FOLLOWER]: Leader election won for term 10
I20251023 02:09:54.144096 19485 raft_consensus.cc:2468] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24706c22e344a48b1f6715bd3115461 in term 10.
I20251023 02:09:54.144459 20397 raft_consensus.cc:697] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 10 LEADER]: Becoming Leader. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Running, Role: LEADER
I20251023 02:09:54.144737 20397 consensus_queue.cc:237] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 9.42, Last appended by leader: 42, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:54.146811 19232 catalog_manager.cc:5649] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 reported cstate change: term changed from 9 to 10, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to d24706c22e344a48b1f6715bd3115461 (127.15.161.197). New cstate: current_term: 10 leader_uuid: "d24706c22e344a48b1f6715bd3115461" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:54.543920 19733 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 10 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 9 index: 42. Preceding OpId from leader: term: 10 index: 43. (index mismatch)
I20251023 02:09:54.544319 20397 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20251023 02:09:54.547356 19485 raft_consensus.cc:1275] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 10 FOLLOWER]: Refusing update from remote peer d24706c22e344a48b1f6715bd3115461: Log matching property violated. Preceding OpId in replica: term: 9 index: 42. Preceding OpId from leader: term: 10 index: 43. (index mismatch)
I20251023 02:09:54.548172 20412 consensus_queue.cc:1048] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20251023 02:09:55.911850 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:55.913997 19810 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808"
dest_uuid: "d24706c22e344a48b1f6715bd3115461"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:43248
I20251023 02:09:55.914186 19810 raft_consensus.cc:606] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:09:55.914561 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:55.914753 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:55.952950 20397 raft_consensus.cc:993] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:55.953081 20424 raft_consensus.cc:1081] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:09:55.953575 19339 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:40250
I20251023 02:09:55.953729 19339 raft_consensus.cc:493] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:55.953810 19339 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:55.955355 19339 raft_consensus.cc:515] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:55.955833 19339 leader_election.cc:290] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:09:55.956149 19810 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:09:55.956149 19408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "390052bc00ba460a83d0c15f6f6b8808" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:09:55.956344 19408 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:55.956341 19810 raft_consensus.cc:3055] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:09:55.956462 19810 raft_consensus.cc:740] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d24706c22e344a48b1f6715bd3115461, State: Running, Role: LEADER
I20251023 02:09:55.956692 19810 consensus_queue.cc:260] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:55.957049 19810 raft_consensus.cc:3060] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:09:55.958853 19408 raft_consensus.cc:2468] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:55.959168 19302 leader_election.cc:304] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, 6d761a2ec1dd4fed941acf04edae065f; no voters:
I20251023 02:09:55.959288 19810 raft_consensus.cc:2468] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 3.
I20251023 02:09:55.959388 20410 raft_consensus.cc:2804] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:09:55.962379 20410 raft_consensus.cc:697] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:55.962580 20410 consensus_queue.cc:237] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:09:55.964401 19232 catalog_manager.cc:5649] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 2 to 3, leader changed from d24706c22e344a48b1f6715bd3115461 (127.15.161.197) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 3 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:56.418268 19408 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:56.418709 20410 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:56.420513 19810 raft_consensus.cc:1275] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 02:09:56.421506 20410 consensus_queue.cc:1048] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 02:09:57.915046 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:57.917146 19339 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:09:57.917311 19339 raft_consensus.cc:606] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:09:57.917725 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:57.917923 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:09:58.328029 20425 raft_consensus.cc:993] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:58.328217 20425 raft_consensus.cc:1081] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:09:58.328699 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37846
I20251023 02:09:58.328843 19485 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:09:58.328914 19485 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:58.330538 19485 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:58.330962 19485 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:09:58.331292 19733 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:09:58.331292 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:09:58.331508 19339 raft_consensus.cc:3055] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 LEADER]: Stepping down as leader of term 5
I20251023 02:09:58.331506 19733 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:58.331617 19339 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:09:58.331848 19339 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:58.332139 19339 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Advancing to term 6
I20251023 02:09:58.333881 19733 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 6.
I20251023 02:09:58.334228 19454 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:09:58.334344 19339 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 6.
I20251023 02:09:58.334528 20411 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20251023 02:09:58.334882 20411 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:09:58.335134 20411 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:09:58.337136 19232 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 5 to 6, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 6 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:09:58.755749 19339 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251023 02:09:58.756273 20411 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:58.760967 19733 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 6 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251023 02:09:58.761456 20449 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 02:09:59.918283 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:09:59.920486 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:09:59.920652 19485 raft_consensus.cc:606] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6d761a2ec1dd4fed941acf04edae065f
I20251023 02:09:59.921057 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:09:59.921247 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:00.178362 20452 raft_consensus.cc:993] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:10:00.178534 20452 raft_consensus.cc:1081] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Signalling peer 6d761a2ec1dd4fed941acf04edae065f to start an election
I20251023 02:10:00.179040 19408 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
from {username='slave'} at 127.0.0.1:35500
I20251023 02:10:00.179199 19408 raft_consensus.cc:493] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:10:00.179308 19408 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:00.180958 19408 raft_consensus.cc:515] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:00.181545 19408 leader_election.cc:290] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:10:00.181922 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:10:00.181922 19809 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "6d761a2ec1dd4fed941acf04edae065f" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:10:00.182152 19485 raft_consensus.cc:3055] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:10:00.182179 19809 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:00.182271 19485 raft_consensus.cc:740] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:10:00.182518 19485 consensus_queue.cc:260] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:00.182834 19485 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:00.184585 19809 raft_consensus.cc:2468] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:10:00.184955 19378 leader_election.cc:304] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d761a2ec1dd4fed941acf04edae065f, d24706c22e344a48b1f6715bd3115461; no voters:
I20251023 02:10:00.185017 19485 raft_consensus.cc:2468] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d761a2ec1dd4fed941acf04edae065f in term 2.
I20251023 02:10:00.185194 20355 raft_consensus.cc:2804] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:00.185554 20355 raft_consensus.cc:697] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Leader. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:10:00.185791 20355 consensus_queue.cc:237] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:00.187860 19232 catalog_manager.cc:5649] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f reported cstate change: term changed from 1 to 2, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194). New cstate: current_term: 2 leader_uuid: "6d761a2ec1dd4fed941acf04edae065f" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:00.777964 19485 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:10:00.778465 20355 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:10:00.781452 19809 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Refusing update from remote peer 6d761a2ec1dd4fed941acf04edae065f: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 02:10:00.781984 20464 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 02:10:01.921531 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:01.924006 19408 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2"
dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:52410
I20251023 02:10:01.924152 19408 raft_consensus.cc:606] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:10:01.924525 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:10:01.924702 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:02.249572 20470 raft_consensus.cc:993] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:10:02.249701 20470 raft_consensus.cc:1081] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:10:02.250156 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37860
I20251023 02:10:02.250288 19485 raft_consensus.cc:493] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:10:02.250347 19485 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:02.252540 19485 raft_consensus.cc:515] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:02.253064 19485 leader_election.cc:290] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194:36623), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:10:02.253352 19408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "6d761a2ec1dd4fed941acf04edae065f"
I20251023 02:10:02.253412 19809 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9973c11f67d54ed2a568e03d04d91ab2" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:10:02.253484 19408 raft_consensus.cc:3055] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:10:02.253548 19408 raft_consensus.cc:740] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d761a2ec1dd4fed941acf04edae065f, State: Running, Role: LEADER
I20251023 02:10:02.253553 19809 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:02.253760 19408 consensus_queue.cc:260] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:02.253973 19408 raft_consensus.cc:3060] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:02.255187 19809 raft_consensus.cc:2468] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 3.
I20251023 02:10:02.255568 19408 raft_consensus.cc:2468] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 3.
I20251023 02:10:02.255546 19452 leader_election.cc:304] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, d24706c22e344a48b1f6715bd3115461; no voters:
I20251023 02:10:02.255807 20452 raft_consensus.cc:2804] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:10:02.256107 20452 raft_consensus.cc:697] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:10:02.256301 20452 consensus_queue.cc:237] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:02.258262 19233 catalog_manager.cc:5649] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 2 to 3, leader changed from 6d761a2ec1dd4fed941acf04edae065f (127.15.161.194) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 3 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:02.818173 19809 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 02:10:02.818670 20452 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:10:02.821693 19408 raft_consensus.cc:1275] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 02:10:02.822885 20452 consensus_queue.cc:1048] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d761a2ec1dd4fed941acf04edae065f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 02:10:03.924991 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:03.927168 19485 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
mode: GRACEFUL
new_leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:37888
I20251023 02:10:03.927333 19485 raft_consensus.cc:606] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Received request to transfer leadership to TS 5cfe6544d1b249e59e921b04cd1c5bcb
I20251023 02:10:03.927783 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:10:03.927982 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:04.438199 20482 raft_consensus.cc:993] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: : Instructing follower 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:10:04.438320 20482 raft_consensus.cc:1081] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Signalling peer 5cfe6544d1b249e59e921b04cd1c5bcb to start an election
I20251023 02:10:04.438858 19339 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
from {username='slave'} at 127.0.0.1:44374
I20251023 02:10:04.438985 19339 raft_consensus.cc:493] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:10:04.439043 19339 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Advancing to term 7
I20251023 02:10:04.440825 19339 raft_consensus.cc:515] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:10:04.441469 19339 leader_election.cc:290] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 7 election: Requested vote from peers 42800175d61f4a9d8a28536807976ad6 (127.15.161.195:38095), e56d3da1a4934bc59b49951222c4ba5f (127.15.161.196:34873)
I20251023 02:10:04.441743 19485 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 7 candidate_status { last_received { term: 6 index: 36 } } ignore_live_leader: true dest_uuid: "42800175d61f4a9d8a28536807976ad6"
I20251023 02:10:04.441802 19733 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b746cb496f464e6c82421b1ab2dd1de0" candidate_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" candidate_term: 7 candidate_status { last_received { term: 6 index: 36 } } ignore_live_leader: true dest_uuid: "e56d3da1a4934bc59b49951222c4ba5f"
I20251023 02:10:04.441888 19485 raft_consensus.cc:3055] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20251023 02:10:04.441888 19733 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 6 FOLLOWER]: Advancing to term 7
I20251023 02:10:04.441968 19485 raft_consensus.cc:740] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:10:04.442203 19485 consensus_queue.cc:260] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 6.36, Last appended by leader: 36, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:10:04.442435 19485 raft_consensus.cc:3060] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Advancing to term 7
I20251023 02:10:04.444139 19733 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 7.
I20251023 02:10:04.444489 19485 raft_consensus.cc:2468] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cfe6544d1b249e59e921b04cd1c5bcb in term 7.
I20251023 02:10:04.444569 19305 leader_election.cc:304] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfe6544d1b249e59e921b04cd1c5bcb, e56d3da1a4934bc59b49951222c4ba5f; no voters:
I20251023 02:10:04.444828 20425 raft_consensus.cc:2804] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 7 FOLLOWER]: Leader election won for term 7
I20251023 02:10:04.445128 20425 raft_consensus.cc:697] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 7 LEADER]: Becoming Leader. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:10:04.445333 20425 consensus_queue.cc:237] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 6.36, Last appended by leader: 36, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } }
I20251023 02:10:04.447161 19232 catalog_manager.cc:5649] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb reported cstate change: term changed from 6 to 7, leader changed from 42800175d61f4a9d8a28536807976ad6 (127.15.161.195) to 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193). New cstate: current_term: 7 leader_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:04.852720 19485 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 6 index: 36. Preceding OpId from leader: term: 7 index: 37. (index mismatch)
I20251023 02:10:04.853165 20425 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:10:04.856343 19733 raft_consensus.cc:1275] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Refusing update from remote peer 5cfe6544d1b249e59e921b04cd1c5bcb: Log matching property violated. Preceding OpId in replica: term: 6 index: 36. Preceding OpId from leader: term: 7 index: 37. (index mismatch)
I20251023 02:10:04.856981 20425 consensus_queue.cc:1048] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d3da1a4934bc59b49951222c4ba5f" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 34873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251023 02:10:05.928351 19290 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:05.930697 19339 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
mode: GRACEFUL
new_leader_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:40234
I20251023 02:10:05.930902 19339 raft_consensus.cc:606] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Received request to transfer leadership to TS 42800175d61f4a9d8a28536807976ad6
I20251023 02:10:05.931324 19290 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 02:10:05.931562 19290 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:06.366673 20498 raft_consensus.cc:993] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: : Instructing follower 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:10:06.366838 20498 raft_consensus.cc:1081] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Signalling peer 42800175d61f4a9d8a28536807976ad6 to start an election
I20251023 02:10:06.367353 19485 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28"
dest_uuid: "42800175d61f4a9d8a28536807976ad6"
from {username='slave'} at 127.0.0.1:37846
I20251023 02:10:06.367527 19485 raft_consensus.cc:493] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:10:06.367614 19485 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:10:06.369805 19485 raft_consensus.cc:515] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:06.370491 19485 leader_election.cc:290] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093), d24706c22e344a48b1f6715bd3115461 (127.15.161.197:46797)
I20251023 02:10:06.370754 19339 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb"
I20251023 02:10:06.370946 19339 raft_consensus.cc:3055] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Stepping down as leader of term 4
I20251023 02:10:06.371042 19339 raft_consensus.cc:740] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5cfe6544d1b249e59e921b04cd1c5bcb, State: Running, Role: LEADER
I20251023 02:10:06.371001 19809 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8ef6d3c01f1e49f386bb4898262f4d28" candidate_uuid: "42800175d61f4a9d8a28536807976ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "d24706c22e344a48b1f6715bd3115461"
I20251023 02:10:06.371183 19809 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:10:06.371272 19339 consensus_queue.cc:260] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:06.371551 19339 raft_consensus.cc:3060] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Advancing to term 5
I20251023 02:10:06.372924 19809 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 5.
I20251023 02:10:06.373163 19339 raft_consensus.cc:2468] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42800175d61f4a9d8a28536807976ad6 in term 5.
I20251023 02:10:06.373301 19452 leader_election.cc:304] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42800175d61f4a9d8a28536807976ad6, d24706c22e344a48b1f6715bd3115461; no voters:
I20251023 02:10:06.373528 20482 raft_consensus.cc:2804] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20251023 02:10:06.373917 20482 raft_consensus.cc:697] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 42800175d61f4a9d8a28536807976ad6, State: Running, Role: LEADER
I20251023 02:10:06.374224 20482 consensus_queue.cc:237] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } }
I20251023 02:10:06.376387 19233 catalog_manager.cc:5649] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 reported cstate change: term changed from 4 to 5, leader changed from 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193) to 42800175d61f4a9d8a28536807976ad6 (127.15.161.195). New cstate: current_term: 5 leader_uuid: "42800175d61f4a9d8a28536807976ad6" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "5cfe6544d1b249e59e921b04cd1c5bcb" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 43093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42800175d61f4a9d8a28536807976ad6" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_rebalancer-test.cc:195: Failure
Expected equality of these values:
0
NumMovesScheduled(leader_idx, BalanceThreadType::LEADER_REBALANCE)
Which is: 1
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_rebalancer-test.cc:575: Failure
Expected: CheckNoLeaderMovesScheduled() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20251023 02:10:06.850675 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:10:06.861127 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:06.861406 16007 tablet_replica.cc:333] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:10:06.861604 16007 raft_consensus.cc:2243] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.861874 16007 raft_consensus.cc:2272] T 24ed6c24dc924a0ab03d84241f9d96ea P 5cfe6544d1b249e59e921b04cd1c5bcb [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.862443 16007 tablet_replica.cc:333] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:10:06.862567 16007 raft_consensus.cc:2243] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.862674 16007 raft_consensus.cc:2272] T 8ef6d3c01f1e49f386bb4898262f4d28 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 5 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.863201 16007 tablet_replica.cc:333] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:10:06.863335 16007 raft_consensus.cc:2243] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.863595 16007 raft_consensus.cc:2272] T 390052bc00ba460a83d0c15f6f6b8808 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.864081 16007 tablet_replica.cc:333] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:10:06.864223 16007 raft_consensus.cc:2243] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.864475 16007 raft_consensus.cc:2272] T 09457d2cf60844348195a77995f7cb88 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.864889 16007 tablet_replica.cc:333] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:10:06.865010 16007 raft_consensus.cc:2243] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.865200 16007 raft_consensus.cc:2272] T 5dab8a517d0a41ceb8ccf4b03d137571 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.865585 16007 tablet_replica.cc:333] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb: stopping tablet replica
I20251023 02:10:06.865700 16007 raft_consensus.cc:2243] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 7 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.865917 16007 raft_consensus.cc:2272] T b746cb496f464e6c82421b1ab2dd1de0 P 5cfe6544d1b249e59e921b04cd1c5bcb [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.883919 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:10:06.888537 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
W20251023 02:10:06.900481 19452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.193:43093: connect: Connection refused (error 111)
W20251023 02:10:06.901403 19452 consensus_peers.cc:597] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 -> Peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Couldn't send request to peer 5cfe6544d1b249e59e921b04cd1c5bcb. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:43093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:06.910360 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:06.910614 16007 tablet_replica.cc:333] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:10:06.910778 16007 raft_consensus.cc:2243] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.911012 16007 raft_consensus.cc:2272] T 38f0952ea765453aaf13876360de864f P 6d761a2ec1dd4fed941acf04edae065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.911506 16007 tablet_replica.cc:333] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:10:06.911621 16007 raft_consensus.cc:2243] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.911782 16007 raft_consensus.cc:2272] T aece3449e9914e6a83067ce373506a58 P 6d761a2ec1dd4fed941acf04edae065f [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.912204 16007 tablet_replica.cc:333] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:10:06.912317 16007 raft_consensus.cc:2243] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.912428 16007 raft_consensus.cc:2272] T 9973c11f67d54ed2a568e03d04d91ab2 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.912907 16007 tablet_replica.cc:333] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:10:06.913028 16007 raft_consensus.cc:2243] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.913344 16007 raft_consensus.cc:2272] T 390052bc00ba460a83d0c15f6f6b8808 P 6d761a2ec1dd4fed941acf04edae065f [term 3 FOLLOWER]: Raft consensus is shut down!
W20251023 02:10:06.913575 19452 consensus_peers.cc:597] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 -> Peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Couldn't send request to peer 5cfe6544d1b249e59e921b04cd1c5bcb. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:43093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:06.913766 16007 tablet_replica.cc:333] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:10:06.913875 16007 raft_consensus.cc:2243] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.914044 16007 raft_consensus.cc:2272] T 4f894b15dfc24915a2e34e4dbdf85121 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.914422 16007 tablet_replica.cc:333] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f: stopping tablet replica
I20251023 02:10:06.914526 16007 raft_consensus.cc:2243] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.914712 16007 raft_consensus.cc:2272] T 09457d2cf60844348195a77995f7cb88 P 6d761a2ec1dd4fed941acf04edae065f [term 4 FOLLOWER]: Raft consensus is shut down!
W20251023 02:10:06.923844 19850 consensus_peers.cc:597] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad -> Peer 5cfe6544d1b249e59e921b04cd1c5bcb (127.15.161.193:43093): Couldn't send request to peer 5cfe6544d1b249e59e921b04cd1c5bcb. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:43093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:06.929845 19809 raft_consensus.cc:1275] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Refusing update from remote peer 42800175d61f4a9d8a28536807976ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 02:10:06.929888 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:10:06.930341 20482 consensus_queue.cc:1048] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24706c22e344a48b1f6715bd3115461" member_type: VOTER last_known_addr { host: "127.15.161.197" port: 46797 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 02:10:06.936019 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:10:06.946761 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:06.946952 16007 tablet_replica.cc:333] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:10:06.947127 16007 raft_consensus.cc:2243] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.947255 16007 raft_consensus.cc:2272] T b746cb496f464e6c82421b1ab2dd1de0 P 42800175d61f4a9d8a28536807976ad6 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.947676 16007 tablet_replica.cc:333] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:10:06.947804 16007 raft_consensus.cc:2243] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.947924 16007 raft_consensus.cc:2272] T 1f28c69fe3504243ad3cb29e0c5e2d93 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.948295 16007 tablet_replica.cc:333] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:10:06.948405 16007 raft_consensus.cc:2243] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 5 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.948671 16007 raft_consensus.cc:2272] T 8ef6d3c01f1e49f386bb4898262f4d28 P 42800175d61f4a9d8a28536807976ad6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.949154 16007 tablet_replica.cc:333] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:10:06.949239 16007 raft_consensus.cc:2243] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.949404 16007 raft_consensus.cc:2272] T 24ed6c24dc924a0ab03d84241f9d96ea P 42800175d61f4a9d8a28536807976ad6 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.964207 16007 tablet_replica.cc:333] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:10:06.964311 16007 raft_consensus.cc:2243] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 3 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.964502 16007 raft_consensus.cc:2272] T 9973c11f67d54ed2a568e03d04d91ab2 P 42800175d61f4a9d8a28536807976ad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.964850 16007 tablet_replica.cc:333] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6: stopping tablet replica
I20251023 02:10:06.964931 16007 raft_consensus.cc:2243] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 10 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.965013 16007 raft_consensus.cc:2272] T 579d6f2275814c5eae528d85a86bf288 P 42800175d61f4a9d8a28536807976ad6 [term 10 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.980752 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:10:06.985219 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:10:06.993800 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:06.993988 16007 tablet_replica.cc:333] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f: stopping tablet replica
I20251023 02:10:06.994148 16007 raft_consensus.cc:2243] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.994340 16007 raft_consensus.cc:2272] T 38f0952ea765453aaf13876360de864f P e56d3da1a4934bc59b49951222c4ba5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.994810 16007 tablet_replica.cc:333] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f: stopping tablet replica
I20251023 02:10:06.994920 16007 raft_consensus.cc:2243] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.995028 16007 raft_consensus.cc:2272] T 1f28c69fe3504243ad3cb29e0c5e2d93 P e56d3da1a4934bc59b49951222c4ba5f [term 5 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.995388 16007 tablet_replica.cc:333] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f: stopping tablet replica
I20251023 02:10:06.995496 16007 raft_consensus.cc:2243] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 10 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.995611 16007 raft_consensus.cc:2272] T 579d6f2275814c5eae528d85a86bf288 P e56d3da1a4934bc59b49951222c4ba5f [term 10 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.995967 16007 tablet_replica.cc:333] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f: stopping tablet replica
I20251023 02:10:06.996071 16007 raft_consensus.cc:2243] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:06.996169 16007 raft_consensus.cc:2272] T b746cb496f464e6c82421b1ab2dd1de0 P e56d3da1a4934bc59b49951222c4ba5f [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.996510 16007 tablet_replica.cc:333] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f: stopping tablet replica
I20251023 02:10:06.996611 16007 raft_consensus.cc:2243] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.996826 16007 raft_consensus.cc:2272] T aece3449e9914e6a83067ce373506a58 P e56d3da1a4934bc59b49951222c4ba5f [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:06.997175 16007 tablet_replica.cc:333] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f: stopping tablet replica
I20251023 02:10:06.997275 16007 raft_consensus.cc:2243] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 LEADER]: Raft consensus shutting down.
I20251023 02:10:06.997457 16007 raft_consensus.cc:2272] T 4f894b15dfc24915a2e34e4dbdf85121 P e56d3da1a4934bc59b49951222c4ba5f [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.012288 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:10:07.015664 16007 tablet_server.cc:178] TabletServer@127.15.161.197:0 shutting down...
I20251023 02:10:07.024104 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:07.024276 16007 tablet_replica.cc:333] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461: stopping tablet replica
I20251023 02:10:07.024412 16007 raft_consensus.cc:2243] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.024564 16007 raft_consensus.cc:2272] T 9973c11f67d54ed2a568e03d04d91ab2 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.025007 16007 tablet_replica.cc:333] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461: stopping tablet replica
I20251023 02:10:07.025118 16007 raft_consensus.cc:2243] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 5 LEADER]: Raft consensus shutting down.
I20251023 02:10:07.025349 16007 raft_consensus.cc:2272] T 1f28c69fe3504243ad3cb29e0c5e2d93 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.025759 16007 tablet_replica.cc:333] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461: stopping tablet replica
I20251023 02:10:07.025871 16007 raft_consensus.cc:2243] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.026021 16007 raft_consensus.cc:2272] T 5dab8a517d0a41ceb8ccf4b03d137571 P d24706c22e344a48b1f6715bd3115461 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.026407 16007 tablet_replica.cc:333] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461: stopping tablet replica
I20251023 02:10:07.026520 16007 raft_consensus.cc:2243] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 10 LEADER]: Raft consensus shutting down.
I20251023 02:10:07.026734 16007 raft_consensus.cc:2272] T 579d6f2275814c5eae528d85a86bf288 P d24706c22e344a48b1f6715bd3115461 [term 10 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.027094 16007 tablet_replica.cc:333] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461: stopping tablet replica
I20251023 02:10:07.027202 16007 raft_consensus.cc:2243] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.027318 16007 raft_consensus.cc:2272] T 390052bc00ba460a83d0c15f6f6b8808 P d24706c22e344a48b1f6715bd3115461 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.027683 16007 tablet_replica.cc:333] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461: stopping tablet replica
I20251023 02:10:07.027788 16007 raft_consensus.cc:2243] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.027967 16007 raft_consensus.cc:2272] T 8ef6d3c01f1e49f386bb4898262f4d28 P d24706c22e344a48b1f6715bd3115461 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.042272 16007 tablet_server.cc:195] TabletServer@127.15.161.197:0 shutdown complete.
I20251023 02:10:07.061444 16007 tablet_server.cc:178] TabletServer@127.15.161.198:0 shutting down...
I20251023 02:10:07.069595 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:07.069819 16007 tablet_replica.cc:333] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad: stopping tablet replica
I20251023 02:10:07.069974 16007 raft_consensus.cc:2243] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.070192 16007 raft_consensus.cc:2272] T 38f0952ea765453aaf13876360de864f P 4a65545dd26a4a018e7a9927513adaad [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.070675 16007 tablet_replica.cc:333] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad: stopping tablet replica
I20251023 02:10:07.070783 16007 raft_consensus.cc:2243] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:10:07.070986 16007 raft_consensus.cc:2272] T 5dab8a517d0a41ceb8ccf4b03d137571 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.071334 16007 tablet_replica.cc:333] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad: stopping tablet replica
I20251023 02:10:07.071421 16007 raft_consensus.cc:2243] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.071542 16007 raft_consensus.cc:2272] T aece3449e9914e6a83067ce373506a58 P 4a65545dd26a4a018e7a9927513adaad [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.071861 16007 tablet_replica.cc:333] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad: stopping tablet replica
I20251023 02:10:07.071941 16007 raft_consensus.cc:2243] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.072065 16007 raft_consensus.cc:2272] T 24ed6c24dc924a0ab03d84241f9d96ea P 4a65545dd26a4a018e7a9927513adaad [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.072384 16007 tablet_replica.cc:333] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad: stopping tablet replica
I20251023 02:10:07.072479 16007 raft_consensus.cc:2243] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.072630 16007 raft_consensus.cc:2272] T 09457d2cf60844348195a77995f7cb88 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.072978 16007 tablet_replica.cc:333] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad: stopping tablet replica
I20251023 02:10:07.073076 16007 raft_consensus.cc:2243] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:07.073212 16007 raft_consensus.cc:2272] T 4f894b15dfc24915a2e34e4dbdf85121 P 4a65545dd26a4a018e7a9927513adaad [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.087071 16007 tablet_server.cc:195] TabletServer@127.15.161.198:0 shutdown complete.
I20251023 02:10:07.090509 16007 master.cc:561] Master@127.15.161.254:33133 shutting down...
I20251023 02:10:07.097260 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:07.097407 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:07.097473 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5a15388250d243a49fbb6e7026c640fe: stopping tablet replica
I20251023 02:10:07.110258 16007 master.cc:583] Master@127.15.161.254:33133 shutdown complete.
I20251023 02:10:07.116073 16007 test_util.cc:183] -----------------------------------------------
I20251023 02:10:07.116113 16007 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761185276310797-16007-0
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer (39736 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingUnstableCluster
I20251023 02:10:07.117623 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:37883
I20251023 02:10:07.118074 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:07.119890 20516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.119995 20517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.120640 20519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:07.126351 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:07.126528 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:07.126572 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:07.126602 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185407126598 us; error 0 us; skew 500 ppm
I20251023 02:10:07.126821 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:07.127969 16007 webserver.cc:492] Webserver started at http://127.15.161.254:32987/ using document root <none> and password file <none>
I20251023 02:10:07.128180 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:07.128235 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:07.128350 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:07.128929 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "83954bb958e748f097019be22d8efe07"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.131165 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:07.132390 20524 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.132637 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:07.132730 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "83954bb958e748f097019be22d8efe07"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.132807 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:07.146746 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:07.147105 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:07.156502 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:37883
I20251023 02:10:07.156517 20575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:37883 every 8 connection(s)
I20251023 02:10:07.157522 20576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:07.161065 20576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07: Bootstrap starting.
I20251023 02:10:07.162084 20576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:07.163308 20576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07: No bootstrap required, opened a new log
I20251023 02:10:07.163763 20576 raft_consensus.cc:359] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83954bb958e748f097019be22d8efe07" member_type: VOTER }
I20251023 02:10:07.163864 20576 raft_consensus.cc:385] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:07.163893 20576 raft_consensus.cc:740] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83954bb958e748f097019be22d8efe07, State: Initialized, Role: FOLLOWER
I20251023 02:10:07.164031 20576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83954bb958e748f097019be22d8efe07" member_type: VOTER }
I20251023 02:10:07.164125 20576 raft_consensus.cc:399] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:10:07.164160 20576 raft_consensus.cc:493] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:10:07.164202 20576 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:07.165671 20576 raft_consensus.cc:515] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83954bb958e748f097019be22d8efe07" member_type: VOTER }
I20251023 02:10:07.165810 20576 leader_election.cc:304] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83954bb958e748f097019be22d8efe07; no voters:
I20251023 02:10:07.166081 20576 leader_election.cc:290] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:10:07.166198 20579 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:07.166563 20579 raft_consensus.cc:697] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 1 LEADER]: Becoming Leader. State: Replica: 83954bb958e748f097019be22d8efe07, State: Running, Role: LEADER
I20251023 02:10:07.166782 20576 sys_catalog.cc:565] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:10:07.166744 20579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83954bb958e748f097019be22d8efe07" member_type: VOTER }
I20251023 02:10:07.167376 20580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83954bb958e748f097019be22d8efe07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83954bb958e748f097019be22d8efe07" member_type: VOTER } }
I20251023 02:10:07.167479 20581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83954bb958e748f097019be22d8efe07. Latest consensus state: current_term: 1 leader_uuid: "83954bb958e748f097019be22d8efe07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83954bb958e748f097019be22d8efe07" member_type: VOTER } }
I20251023 02:10:07.167585 20580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:07.167611 20581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:07.168697 20584 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:10:07.169973 20584 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:10:07.171432 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:10:07.172523 20584 catalog_manager.cc:1357] Generated new cluster ID: 8d266072f79d4a59a330cdc3a06e462d
I20251023 02:10:07.172587 20584 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:10:07.188024 20584 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:10:07.188740 20584 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:10:07.200491 20584 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07: Generated new TSK 0
I20251023 02:10:07.200685 20584 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:10:07.203711 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:07.205788 20597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.205987 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.206386 20600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:07.206802 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:07.206998 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:07.207044 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:07.207075 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185407207071 us; error 0 us; skew 500 ppm
I20251023 02:10:07.207194 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:07.208187 16007 webserver.cc:492] Webserver started at http://127.15.161.193:43837/ using document root <none> and password file <none>
I20251023 02:10:07.208379 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:07.208451 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:07.208568 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:07.209157 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "44dc70fd17da43b4af8380d6acae2cf0"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.211194 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:07.212407 20605 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.212723 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:07.212803 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "44dc70fd17da43b4af8380d6acae2cf0"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.212872 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:07.222882 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:07.223390 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:07.224139 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:07.225116 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:07.225186 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.225260 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:07.225303 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.241539 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:45245
I20251023 02:10:07.241559 20667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:45245 every 8 connection(s)
I20251023 02:10:07.255785 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:07.258163 20672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.258219 20673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.261722 20675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:07.261930 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:07.262914 20668 heartbeater.cc:344] Connected to a master server at 127.15.161.254:37883
I20251023 02:10:07.263060 20668 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:07.263428 20668 heartbeater.cc:507] Master 127.15.161.254:37883 requested a full tablet report, sending...
I20251023 02:10:07.264331 20541 ts_manager.cc:194] Registered new tserver with Master: 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245)
I20251023 02:10:07.265515 20541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40330
I20251023 02:10:07.280165 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:07.280231 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:07.280251 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185407280249 us; error 0 us; skew 500 ppm
I20251023 02:10:07.280366 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:07.281476 16007 webserver.cc:492] Webserver started at http://127.15.161.194:44643/ using document root <none> and password file <none>
I20251023 02:10:07.281672 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:07.281728 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:07.281844 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:07.282454 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.284492 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:07.285676 20680 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.285919 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:07.285998 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.286073 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:07.307518 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:07.308037 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:07.308531 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:07.309300 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:07.309348 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.309456 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:07.309499 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.320850 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:37169
I20251023 02:10:07.320876 20742 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:37169 every 8 connection(s)
I20251023 02:10:07.322137 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:07.324435 20746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.324882 20747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:07.325004 20749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:07.325613 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:07.325814 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:07.325858 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:07.325896 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185407325892 us; error 0 us; skew 500 ppm
I20251023 02:10:07.326010 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:07.327250 16007 webserver.cc:492] Webserver started at http://127.15.161.195:35989/ using document root <none> and password file <none>
I20251023 02:10:07.327468 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:07.327549 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:07.327605 20743 heartbeater.cc:344] Connected to a master server at 127.15.161.254:37883
I20251023 02:10:07.327646 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:07.327708 20743 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:07.327968 20743 heartbeater.cc:507] Master 127.15.161.254:37883 requested a full tablet report, sending...
I20251023 02:10:07.328250 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.328748 20541 ts_manager.cc:194] Registered new tserver with Master: 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194:37169)
I20251023 02:10:07.329666 20541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40344
I20251023 02:10:07.330884 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:10:07.332118 20754 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.332398 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:07.332468 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
format_stamp: "Formatted at 2025-10-23 02:10:07 on dist-test-slave-bfbt"
I20251023 02:10:07.332546 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:07.358623 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:07.359081 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:07.359566 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:07.360268 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:07.360312 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.360361 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:07.360404 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:07.372713 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:43429
I20251023 02:10:07.372736 20816 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:43429 every 8 connection(s)
I20251023 02:10:07.377981 20817 heartbeater.cc:344] Connected to a master server at 127.15.161.254:37883
I20251023 02:10:07.378094 20817 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:07.378424 20817 heartbeater.cc:507] Master 127.15.161.254:37883 requested a full tablet report, sending...
I20251023 02:10:07.379174 20541 ts_manager.cc:194] Registered new tserver with Master: 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:07.379832 20541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40354
I20251023 02:10:07.380168 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006717474s
I20251023 02:10:08.273490 20668 heartbeater.cc:499] Master 127.15.161.254:37883 was elected leader, sending a full tablet report...
I20251023 02:10:08.330708 20743 heartbeater.cc:499] Master 127.15.161.254:37883 was elected leader, sending a full tablet report...
I20251023 02:10:08.380873 20817 heartbeater.cc:499] Master 127.15.161.254:37883 was elected leader, sending a full tablet report...
I20251023 02:10:08.407039 16007 test_util.cc:276] Using random seed: -291056133
I20251023 02:10:08.418195 20541 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40364:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:10:08.418813 20541 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:10:08.435712 20782 tablet_service.cc:1505] Processing CreateTablet for tablet 1ef0cb65766841289aaff11ae60a23f0 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 02:10:08.435796 20781 tablet_service.cc:1505] Processing CreateTablet for tablet e710f2a72300483b981e8ca682fdd9c9 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 02:10:08.435804 20780 tablet_service.cc:1505] Processing CreateTablet for tablet a0d08fdc59974dfcb30b8b370a858a54 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 02:10:08.436120 20782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef0cb65766841289aaff11ae60a23f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.436293 20780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0d08fdc59974dfcb30b8b370a858a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.437103 20633 tablet_service.cc:1505] Processing CreateTablet for tablet 1ef0cb65766841289aaff11ae60a23f0 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 02:10:08.437414 20708 tablet_service.cc:1505] Processing CreateTablet for tablet 1ef0cb65766841289aaff11ae60a23f0 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 02:10:08.437741 20708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef0cb65766841289aaff11ae60a23f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.440603 20781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e710f2a72300483b981e8ca682fdd9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.440829 20837 tablet_bootstrap.cc:492] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea: Bootstrap starting.
I20251023 02:10:08.437667 20631 tablet_service.cc:1505] Processing CreateTablet for tablet a0d08fdc59974dfcb30b8b370a858a54 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 02:10:08.442019 20632 tablet_service.cc:1505] Processing CreateTablet for tablet e710f2a72300483b981e8ca682fdd9c9 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 02:10:08.437428 20633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef0cb65766841289aaff11ae60a23f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.442461 20837 tablet_bootstrap.cc:654] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.443089 20632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e710f2a72300483b981e8ca682fdd9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.447424 20631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0d08fdc59974dfcb30b8b370a858a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.449498 20840 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Bootstrap starting.
I20251023 02:10:08.449666 20707 tablet_service.cc:1505] Processing CreateTablet for tablet e710f2a72300483b981e8ca682fdd9c9 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 02:10:08.449978 20707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e710f2a72300483b981e8ca682fdd9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.452466 20840 tablet_bootstrap.cc:654] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.454830 20840 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: No bootstrap required, opened a new log
I20251023 02:10:08.455008 20840 ts_tablet_manager.cc:1403] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251023 02:10:08.455502 20707 tablet_service.cc:1505] Processing CreateTablet for tablet a0d08fdc59974dfcb30b8b370a858a54 (DEFAULT_TABLE table=test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 02:10:08.455855 20707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0d08fdc59974dfcb30b8b370a858a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:08.455859 20840 raft_consensus.cc:359] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.456151 20837 tablet_bootstrap.cc:492] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea: No bootstrap required, opened a new log
I20251023 02:10:08.456255 20840 raft_consensus.cc:385] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.456362 20840 raft_consensus.cc:740] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.456348 20837 ts_tablet_manager.cc:1403] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.004s
I20251023 02:10:08.456678 20840 consensus_queue.cc:260] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.457408 20837 raft_consensus.cc:359] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.457612 20837 raft_consensus.cc:385] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.457743 20837 raft_consensus.cc:740] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d823b4f54d49d7a5fa5bba42dd61ea, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.458082 20840 ts_tablet_manager.cc:1434] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251023 02:10:08.458114 20839 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Bootstrap starting.
I20251023 02:10:08.458385 20840 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Bootstrap starting.
I20251023 02:10:08.459834 20839 tablet_bootstrap.cc:654] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.459834 20840 tablet_bootstrap.cc:654] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.461531 20837 consensus_queue.cc:260] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.462314 20837 ts_tablet_manager.cc:1434] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251023 02:10:08.462553 20837 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea: Bootstrap starting.
I20251023 02:10:08.464569 20837 tablet_bootstrap.cc:654] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.466341 20837 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea: No bootstrap required, opened a new log
I20251023 02:10:08.466421 20840 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: No bootstrap required, opened a new log
I20251023 02:10:08.466467 20837 ts_tablet_manager.cc:1403] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:08.466598 20840 ts_tablet_manager.cc:1403] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20251023 02:10:08.466763 20839 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: No bootstrap required, opened a new log
I20251023 02:10:08.466862 20839 ts_tablet_manager.cc:1403] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20251023 02:10:08.466980 20837 raft_consensus.cc:359] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.467080 20837 raft_consensus.cc:385] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.467149 20837 raft_consensus.cc:740] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d823b4f54d49d7a5fa5bba42dd61ea, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.467227 20840 raft_consensus.cc:359] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.467314 20837 consensus_queue.cc:260] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.467453 20840 raft_consensus.cc:385] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.467443 20839 raft_consensus.cc:359] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.467590 20840 raft_consensus.cc:740] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.467669 20839 raft_consensus.cc:385] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.467761 20837 ts_tablet_manager.cc:1434] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:08.467760 20839 raft_consensus.cc:740] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44dc70fd17da43b4af8380d6acae2cf0, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.467854 20840 consensus_queue.cc:260] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.468071 20837 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: Bootstrap starting.
I20251023 02:10:08.468129 20839 consensus_queue.cc:260] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.469650 20837 tablet_bootstrap.cc:654] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.470737 20840 ts_tablet_manager.cc:1434] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251023 02:10:08.471799 20840 tablet_bootstrap.cc:492] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Bootstrap starting.
I20251023 02:10:08.472045 20839 ts_tablet_manager.cc:1434] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20251023 02:10:08.472278 20839 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: Bootstrap starting.
I20251023 02:10:08.472831 20837 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: No bootstrap required, opened a new log
I20251023 02:10:08.472967 20837 ts_tablet_manager.cc:1403] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251023 02:10:08.473634 20837 raft_consensus.cc:359] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.473804 20837 raft_consensus.cc:385] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.473842 20839 tablet_bootstrap.cc:654] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.473888 20837 raft_consensus.cc:740] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d823b4f54d49d7a5fa5bba42dd61ea, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.474151 20840 tablet_bootstrap.cc:654] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.474198 20837 consensus_queue.cc:260] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.476804 20837 ts_tablet_manager.cc:1434] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251023 02:10:08.479233 20839 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: No bootstrap required, opened a new log
I20251023 02:10:08.479317 20839 ts_tablet_manager.cc:1403] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20251023 02:10:08.479728 20839 raft_consensus.cc:359] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.479830 20839 raft_consensus.cc:385] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.479882 20839 raft_consensus.cc:740] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44dc70fd17da43b4af8380d6acae2cf0, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.479964 20842 raft_consensus.cc:493] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:08.480033 20839 consensus_queue.cc:260] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.480074 20842 raft_consensus.cc:515] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.480366 20839 ts_tablet_manager.cc:1434] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:08.480573 20839 tablet_bootstrap.cc:492] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0: Bootstrap starting.
I20251023 02:10:08.480700 20842 leader_election.cc:290] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:08.481562 20840 tablet_bootstrap.cc:492] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: No bootstrap required, opened a new log
I20251023 02:10:08.481711 20840 ts_tablet_manager.cc:1403] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20251023 02:10:08.482410 20839 tablet_bootstrap.cc:654] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:08.482352 20840 raft_consensus.cc:359] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.482548 20840 raft_consensus.cc:385] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.482626 20840 raft_consensus.cc:740] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.482786 20840 consensus_queue.cc:260] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.483942 20840 ts_tablet_manager.cc:1434] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:08.485101 20839 tablet_bootstrap.cc:492] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0: No bootstrap required, opened a new log
I20251023 02:10:08.485261 20839 ts_tablet_manager.cc:1403] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 02:10:08.485901 20839 raft_consensus.cc:359] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.486038 20839 raft_consensus.cc:385] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:08.486176 20839 raft_consensus.cc:740] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44dc70fd17da43b4af8380d6acae2cf0, State: Initialized, Role: FOLLOWER
I20251023 02:10:08.486348 20839 consensus_queue.cc:260] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.486850 20839 ts_tablet_manager.cc:1434] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:08.488143 20842 raft_consensus.cc:493] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:08.488256 20842 raft_consensus.cc:515] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.488773 20842 leader_election.cc:290] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:08.489804 20643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0" is_pre_election: true
I20251023 02:10:08.490002 20643 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 0.
I20251023 02:10:08.490293 20642 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef0cb65766841289aaff11ae60a23f0" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0" is_pre_election: true
I20251023 02:10:08.490393 20684 leader_election.cc:304] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:08.490478 20642 raft_consensus.cc:2468] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 0.
I20251023 02:10:08.490593 20842 raft_consensus.cc:2804] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:08.490695 20842 raft_consensus.cc:493] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:08.490770 20842 raft_consensus.cc:3060] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.490788 20684 leader_election.cc:304] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:08.493252 20845 raft_consensus.cc:493] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:08.493330 20845 raft_consensus.cc:515] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.493876 20845 leader_election.cc:290] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194:37169)
I20251023 02:10:08.493932 20842 raft_consensus.cc:515] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.494638 20842 leader_election.cc:290] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 election: Requested vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:08.494784 20842 raft_consensus.cc:2804] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:08.494840 20842 raft_consensus.cc:493] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:08.494892 20842 raft_consensus.cc:3060] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.496255 20842 raft_consensus.cc:515] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.496678 20842 leader_election.cc:290] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 election: Requested vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:08.499848 20845 raft_consensus.cc:493] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:08.499984 20845 raft_consensus.cc:515] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.500562 20845 leader_election.cc:290] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194:37169), 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245)
I20251023 02:10:08.500659 20792 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" is_pre_election: true
I20251023 02:10:08.500792 20792 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 0.
I20251023 02:10:08.500993 20790 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
I20251023 02:10:08.501127 20790 raft_consensus.cc:3060] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.503487 20790 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1.
I20251023 02:10:08.503715 20789 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef0cb65766841289aaff11ae60a23f0" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
I20251023 02:10:08.503814 20682 leader_election.cc:304] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d823b4f54d49d7a5fa5bba42dd61ea, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:08.503821 20789 raft_consensus.cc:3060] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.504020 20854 raft_consensus.cc:2804] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:08.504145 20854 raft_consensus.cc:697] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Becoming Leader. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Running, Role: LEADER
I20251023 02:10:08.504359 20854 consensus_queue.cc:237] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.505916 20643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef0cb65766841289aaff11ae60a23f0" candidate_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0" is_pre_election: true
I20251023 02:10:08.506047 20643 raft_consensus.cc:2468] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d823b4f54d49d7a5fa5bba42dd61ea in term 0.
W20251023 02:10:08.506165 20669 tablet.cc:2378] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 02:10:08.506261 20789 raft_consensus.cc:2468] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1.
I20251023 02:10:08.506474 20758 leader_election.cc:304] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 67d823b4f54d49d7a5fa5bba42dd61ea; no voters:
I20251023 02:10:08.506558 20682 leader_election.cc:304] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d823b4f54d49d7a5fa5bba42dd61ea, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:08.500775 20791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef0cb65766841289aaff11ae60a23f0" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" is_pre_election: true
I20251023 02:10:08.506683 20845 raft_consensus.cc:2764] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20251023 02:10:08.506747 20642 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0" is_pre_election: true
I20251023 02:10:08.506812 20791 raft_consensus.cc:2376] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1. Re-sending same reply.
I20251023 02:10:08.506870 20642 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d823b4f54d49d7a5fa5bba42dd61ea in term 0.
I20251023 02:10:08.507046 20854 raft_consensus.cc:2804] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:08.507133 20854 raft_consensus.cc:697] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Becoming Leader. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Running, Role: LEADER
I20251023 02:10:08.507184 20758 leader_election.cc:304] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 67d823b4f54d49d7a5fa5bba42dd61ea; no voters:
I20251023 02:10:08.507320 20854 consensus_queue.cc:237] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.507304 20539 catalog_manager.cc:5649] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194). New cstate: current_term: 1 leader_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:08.508339 20642 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0"
I20251023 02:10:08.508538 20642 raft_consensus.cc:3060] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.510320 20845 raft_consensus.cc:2764] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20251023 02:10:08.511272 20718 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" is_pre_election: true
I20251023 02:10:08.511356 20717 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef0cb65766841289aaff11ae60a23f0" candidate_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" is_pre_election: true
I20251023 02:10:08.511611 20642 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1.
I20251023 02:10:08.512609 20842 raft_consensus.cc:493] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:08.512622 20642 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef0cb65766841289aaff11ae60a23f0" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0"
I20251023 02:10:08.512845 20642 raft_consensus.cc:3060] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.512787 20842 raft_consensus.cc:515] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.513581 20842 leader_election.cc:290] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:08.513670 20791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" is_pre_election: true
I20251023 02:10:08.513670 20643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0" is_pre_election: true
I20251023 02:10:08.513831 20791 raft_consensus.cc:2468] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 0.
I20251023 02:10:08.513831 20643 raft_consensus.cc:2468] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 0.
I20251023 02:10:08.514173 20684 leader_election.cc:304] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:08.514369 20842 raft_consensus.cc:2804] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:08.514477 20842 raft_consensus.cc:493] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:08.514577 20842 raft_consensus.cc:3060] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.514495 20539 catalog_manager.cc:5649] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194). New cstate: current_term: 1 leader_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:08.515249 20642 raft_consensus.cc:2468] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1.
I20251023 02:10:08.516798 20842 raft_consensus.cc:515] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.517165 20842 leader_election.cc:290] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 election: Requested vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:08.517380 20642 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0"
I20251023 02:10:08.517396 20791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54" candidate_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
I20251023 02:10:08.517510 20642 raft_consensus.cc:3060] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.517514 20791 raft_consensus.cc:3060] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:08.519173 20642 raft_consensus.cc:2468] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1.
I20251023 02:10:08.519241 20791 raft_consensus.cc:2468] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867e9ea72c3a4e22abbaaa4cc28e09bd in term 1.
I20251023 02:10:08.519464 20684 leader_election.cc:304] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:08.519657 20842 raft_consensus.cc:2804] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:08.519783 20842 raft_consensus.cc:697] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Becoming Leader. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Running, Role: LEADER
I20251023 02:10:08.519953 20842 consensus_queue.cc:237] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:08.521718 20539 catalog_manager.cc:5649] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194). New cstate: current_term: 1 leader_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:08.532084 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:08.534349 20860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:08.534518 20861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:08.534987 20863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:08.535501 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:08.535720 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:08.535768 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:08.535799 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185408535796 us; error 0 us; skew 500 ppm
I20251023 02:10:08.535933 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:08.536912 16007 webserver.cc:492] Webserver started at http://127.15.161.196:35999/ using document root <none> and password file <none>
I20251023 02:10:08.537077 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:08.537123 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:08.537228 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:08.537707 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "bf103d0fdc514c009863847d63610dff"
format_stamp: "Formatted at 2025-10-23 02:10:08 on dist-test-slave-bfbt"
I20251023 02:10:08.540025 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:08.541450 20868 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:08.541733 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:08.541814 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "bf103d0fdc514c009863847d63610dff"
format_stamp: "Formatted at 2025-10-23 02:10:08 on dist-test-slave-bfbt"
I20251023 02:10:08.541893 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:08.553042 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:08.553437 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:08.553855 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:08.554597 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:08.554646 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:08.554728 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:08.554773 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:08.565658 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:41693
I20251023 02:10:08.565680 20930 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:41693 every 8 connection(s)
I20251023 02:10:08.571179 20931 heartbeater.cc:344] Connected to a master server at 127.15.161.254:37883
I20251023 02:10:08.571272 20931 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:08.571517 20931 heartbeater.cc:507] Master 127.15.161.254:37883 requested a full tablet report, sending...
I20251023 02:10:08.572137 20539 ts_manager.cc:194] Registered new tserver with Master: bf103d0fdc514c009863847d63610dff (127.15.161.196:41693)
I20251023 02:10:08.572793 20539 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40376
I20251023 02:10:08.585175 20842 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:10:08.587323 20854 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20251023 02:10:08.587379 20854 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20251023 02:10:08.607236 20842 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:08.614413 20842 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:08.618676 20842 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:08.621650 20842 consensus_queue.cc:1048] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:08.633493 20842 consensus_queue.cc:1048] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:09.171380 20594 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:09.177784 20717 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54"
dest_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd"
mode: GRACEFUL
new_leader_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
from {username='slave'} at 127.0.0.1:35930
I20251023 02:10:09.177966 20717 raft_consensus.cc:606] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Received request to transfer leadership to TS 67d823b4f54d49d7a5fa5bba42dd61ea
I20251023 02:10:09.178721 20718 consensus_queue.cc:237] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } }
I20251023 02:10:09.178992 20717 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9"
dest_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd"
mode: GRACEFUL
new_leader_uuid: "44dc70fd17da43b4af8380d6acae2cf0"
from {username='slave'} at 127.0.0.1:35930
I20251023 02:10:09.179117 20717 raft_consensus.cc:606] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Received request to transfer leadership to TS 44dc70fd17da43b4af8380d6acae2cf0
I20251023 02:10:09.179431 20594 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251023 02:10:09.179536 20594 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:09.180454 20642 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:09.180454 20789 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:09.180939 20945 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:09.181272 20937 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:09.183620 20642 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
I20251023 02:10:09.183908 20789 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
I20251023 02:10:09.188724 20539 catalog_manager.cc:5649] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea reported cstate change: config changed from index -1 to 2, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
W20251023 02:10:09.192137 20681 consensus_peers.cc:597] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd -> Peer bf103d0fdc514c009863847d63610dff (127.15.161.196:41693): Couldn't send request to peer bf103d0fdc514c009863847d63610dff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ef0cb65766841289aaff11ae60a23f0. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:09.240196 20937 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
W20251023 02:10:09.244351 20589 auto_rebalancer.cc:254] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
W20251023 02:10:09.246430 20589 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet a0d08fdc59974dfcb30b8b370a858a54, TS 867e9ea72c3a4e22abbaaa4cc28e09bd -> TS bf103d0fdc514c009863847d63610dff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
W20251023 02:10:09.246496 20589 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet a0d08fdc59974dfcb30b8b370a858a54, TS 867e9ea72c3a4e22abbaaa4cc28e09bd -> TS bf103d0fdc514c009863847d63610dff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.276473 20945 raft_consensus.cc:993] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: : Instructing follower 67d823b4f54d49d7a5fa5bba42dd61ea to start an election
I20251023 02:10:09.276615 20945 raft_consensus.cc:1081] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Signalling peer 67d823b4f54d49d7a5fa5bba42dd61ea to start an election
I20251023 02:10:09.277081 20789 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54"
dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
from {username='slave'} at 127.0.0.1:49696
I20251023 02:10:09.277236 20789 raft_consensus.cc:493] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:10:09.277333 20789 raft_consensus.cc:3060] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:09.279378 20789 raft_consensus.cc:515] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.279891 20789 leader_election.cc:290] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [CANDIDATE]: Term 2 election: Requested vote from peers 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245), 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194:37169)
I20251023 02:10:09.280119 20642 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54" candidate_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0"
I20251023 02:10:09.280261 20642 raft_consensus.cc:3060] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:09.280349 20718 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0d08fdc59974dfcb30b8b370a858a54" candidate_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd"
I20251023 02:10:09.281162 20718 raft_consensus.cc:3055] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:10:09.281242 20718 raft_consensus.cc:740] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Running, Role: LEADER
I20251023 02:10:09.281440 20718 consensus_queue.cc:260] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.281728 20718 raft_consensus.cc:3060] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:09.282536 20642 raft_consensus.cc:2468] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d823b4f54d49d7a5fa5bba42dd61ea in term 2.
I20251023 02:10:09.282838 20758 leader_election.cc:304] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 67d823b4f54d49d7a5fa5bba42dd61ea; no voters:
I20251023 02:10:09.283047 20952 raft_consensus.cc:2804] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:09.283434 20952 raft_consensus.cc:697] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 LEADER]: Becoming Leader. State: Replica: 67d823b4f54d49d7a5fa5bba42dd61ea, State: Running, Role: LEADER
I20251023 02:10:09.283684 20952 consensus_queue.cc:237] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.283887 20718 raft_consensus.cc:2468] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d823b4f54d49d7a5fa5bba42dd61ea in term 2.
I20251023 02:10:09.286185 20959 ts_tablet_manager.cc:933] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: Initiating tablet copy from peer 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194:37169)
I20251023 02:10:09.286726 20959 tablet_copy_client.cc:323] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:37169
I20251023 02:10:09.292856 20540 catalog_manager.cc:5649] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea reported cstate change: term changed from 1 to 2, leader changed from 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194) to 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195). New cstate: current_term: 2 leader_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:09.293628 20728 tablet_copy_service.cc:140] P 867e9ea72c3a4e22abbaaa4cc28e09bd: Received BeginTabletCopySession request for tablet 1ef0cb65766841289aaff11ae60a23f0 from peer bf103d0fdc514c009863847d63610dff ({username='slave'} at 127.0.0.1:35948)
I20251023 02:10:09.293747 20728 tablet_copy_service.cc:161] P 867e9ea72c3a4e22abbaaa4cc28e09bd: Beginning new tablet copy session on tablet 1ef0cb65766841289aaff11ae60a23f0 from peer bf103d0fdc514c009863847d63610dff at {username='slave'} at 127.0.0.1:35948: session id = bf103d0fdc514c009863847d63610dff-1ef0cb65766841289aaff11ae60a23f0
I20251023 02:10:09.295706 20728 tablet_copy_source_session.cc:215] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:09.296625 20959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef0cb65766841289aaff11ae60a23f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:09.301745 20959 tablet_copy_client.cc:806] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:09.301925 20959 tablet_copy_client.cc:670] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:09.303105 20959 tablet_copy_client.cc:538] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:09.305130 20959 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: Bootstrap starting.
I20251023 02:10:09.310354 20959 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:09.310626 20959 tablet_bootstrap.cc:492] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: Bootstrap complete.
I20251023 02:10:09.310824 20959 ts_tablet_manager.cc:1403] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: Time spent bootstrapping tablet: real 0.006s user 0.007s sys 0.000s
I20251023 02:10:09.311409 20959 raft_consensus.cc:359] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } }
I20251023 02:10:09.311559 20959 raft_consensus.cc:740] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bf103d0fdc514c009863847d63610dff, State: Initialized, Role: LEARNER
I20251023 02:10:09.311769 20959 consensus_queue.cc:260] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } }
I20251023 02:10:09.311980 20842 raft_consensus.cc:993] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: : Instructing follower 44dc70fd17da43b4af8380d6acae2cf0 to start an election
I20251023 02:10:09.312125 20842 raft_consensus.cc:1081] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Signalling peer 44dc70fd17da43b4af8380d6acae2cf0 to start an election
I20251023 02:10:09.312412 20959 ts_tablet_manager.cc:1434] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:09.312448 20931 heartbeater.cc:499] Master 127.15.161.254:37883 was elected leader, sending a full tablet report...
I20251023 02:10:09.312887 20728 tablet_copy_service.cc:342] P 867e9ea72c3a4e22abbaaa4cc28e09bd: Request end of tablet copy session bf103d0fdc514c009863847d63610dff-1ef0cb65766841289aaff11ae60a23f0 received from {username='slave'} at 127.0.0.1:35948
I20251023 02:10:09.313000 20728 tablet_copy_service.cc:434] P 867e9ea72c3a4e22abbaaa4cc28e09bd: ending tablet copy session bf103d0fdc514c009863847d63610dff-1ef0cb65766841289aaff11ae60a23f0 on tablet 1ef0cb65766841289aaff11ae60a23f0 with peer bf103d0fdc514c009863847d63610dff
I20251023 02:10:09.313050 20642 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9"
dest_uuid: "44dc70fd17da43b4af8380d6acae2cf0"
from {username='slave'} at 127.0.0.1:52880
I20251023 02:10:09.313210 20642 raft_consensus.cc:493] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 02:10:09.313318 20642 raft_consensus.cc:3060] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:09.315323 20642 raft_consensus.cc:515] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.315946 20642 leader_election.cc:290] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [CANDIDATE]: Term 2 election: Requested vote from peers 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194:37169), 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429)
I20251023 02:10:09.323348 20718 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "44dc70fd17da43b4af8380d6acae2cf0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd"
I20251023 02:10:09.323660 20718 raft_consensus.cc:3055] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Stepping down as leader of term 1
I20251023 02:10:09.323801 20718 raft_consensus.cc:740] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 867e9ea72c3a4e22abbaaa4cc28e09bd, State: Running, Role: LEADER
I20251023 02:10:09.324327 20718 consensus_queue.cc:260] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.324682 20718 raft_consensus.cc:3060] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:09.328153 20718 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44dc70fd17da43b4af8380d6acae2cf0 in term 2.
I20251023 02:10:09.328508 20606 leader_election.cc:304] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44dc70fd17da43b4af8380d6acae2cf0, 867e9ea72c3a4e22abbaaa4cc28e09bd; no voters:
I20251023 02:10:09.328787 20953 raft_consensus.cc:2804] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:09.329183 20953 raft_consensus.cc:697] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 LEADER]: Becoming Leader. State: Replica: 44dc70fd17da43b4af8380d6acae2cf0, State: Running, Role: LEADER
I20251023 02:10:09.329428 20953 consensus_queue.cc:237] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } }
I20251023 02:10:09.331251 20789 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e710f2a72300483b981e8ca682fdd9c9" candidate_uuid: "44dc70fd17da43b4af8380d6acae2cf0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea"
I20251023 02:10:09.331214 20540 catalog_manager.cc:5649] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 reported cstate change: term changed from 1 to 2, leader changed from 867e9ea72c3a4e22abbaaa4cc28e09bd (127.15.161.194) to 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193). New cstate: current_term: 2 leader_uuid: "44dc70fd17da43b4af8380d6acae2cf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:09.331414 20789 raft_consensus.cc:3060] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:09.333534 20789 raft_consensus.cc:2468] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44dc70fd17da43b4af8380d6acae2cf0 in term 2.
I20251023 02:10:09.365061 20642 raft_consensus.cc:1275] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Refusing update from remote peer 67d823b4f54d49d7a5fa5bba42dd61ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:09.365473 20952 consensus_queue.cc:1048] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:09.369607 20717 raft_consensus.cc:1275] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Refusing update from remote peer 67d823b4f54d49d7a5fa5bba42dd61ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:09.370035 20952 consensus_queue.cc:1048] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:09.408768 20906 raft_consensus.cc:1217] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 02:10:09.412751 20718 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:09.413156 20953 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:09.442003 20789 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:09.442480 20953 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:09.557698 20945 raft_consensus.cc:1064] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: attempting to promote NON_VOTER bf103d0fdc514c009863847d63610dff to VOTER
I20251023 02:10:09.558486 20945 consensus_queue.cc:237] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } }
I20251023 02:10:09.560096 20789 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:09.560238 20642 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:09.560534 20956 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:09.560765 20945 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:09.563318 20789 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.567030 20541 catalog_manager.cc:5649] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea reported cstate change: config changed from index 2 to 3, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.570562 20906 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 LEARNER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:09.570951 20945 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:09.572190 20906 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.630949 20956 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.634585 20642 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } attrs { replace: true } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.656773 20717 consensus_queue.cc:237] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } }
I20251023 02:10:09.658238 20906 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 FOLLOWER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:09.658353 20789 raft_consensus.cc:1275] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Refusing update from remote peer 867e9ea72c3a4e22abbaaa4cc28e09bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:09.658800 20957 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:09.659036 20937 consensus_queue.cc:1048] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:09.661201 20789 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.661250 20906 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.664983 20538 catalog_manager.cc:5649] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff reported cstate change: config changed from index 3 to 4, VOTER 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.675173 20633 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef0cb65766841289aaff11ae60a23f0 with delete_type TABLET_DATA_TOMBSTONED (TS 44dc70fd17da43b4af8380d6acae2cf0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52870
I20251023 02:10:09.675755 20974 tablet_replica.cc:333] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: stopping tablet replica
I20251023 02:10:09.675999 20974 raft_consensus.cc:2243] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:09.676203 20974 raft_consensus.cc:2272] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:09.676973 20974 ts_tablet_manager.cc:1916] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:10:09.682610 20974 ts_tablet_manager.cc:1929] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251023 02:10:09.682757 20974 log.cc:1199] T 1ef0cb65766841289aaff11ae60a23f0 P 44dc70fd17da43b4af8380d6acae2cf0: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/1ef0cb65766841289aaff11ae60a23f0
I20251023 02:10:09.683331 20528 catalog_manager.cc:4985] TS 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245): tablet 1ef0cb65766841289aaff11ae60a23f0 (table test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]) successfully deleted
I20251023 02:10:09.860694 20945 raft_consensus.cc:2955] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:09.863076 20525 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 1ef0cb65766841289aaff11ae60a23f0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:10.870803 20642 consensus_queue.cc:237] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } }
I20251023 02:10:10.873126 20717 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:10:10.873149 20789 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:10:10.873581 20978 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:10.873816 20972 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:10.876763 20789 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
W20251023 02:10:10.878651 20606 consensus_peers.cc:597] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 -> Peer bf103d0fdc514c009863847d63610dff (127.15.161.196:41693): Couldn't send request to peer bf103d0fdc514c009863847d63610dff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e710f2a72300483b981e8ca682fdd9c9. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:10.880234 20717 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
I20251023 02:10:10.880888 20539 catalog_manager.cc:5649] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea reported cstate change: config changed from index -1 to 3, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New cstate: current_term: 2 leader_uuid: "44dc70fd17da43b4af8380d6acae2cf0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
I20251023 02:10:10.950440 20978 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER bf103d0fdc514c009863847d63610dff (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } } }
I20251023 02:10:10.963897 20989 ts_tablet_manager.cc:933] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: Initiating tablet copy from peer 44dc70fd17da43b4af8380d6acae2cf0 (127.15.161.193:45245)
I20251023 02:10:10.964447 20989 tablet_copy_client.cc:323] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:45245
I20251023 02:10:10.969664 20653 tablet_copy_service.cc:140] P 44dc70fd17da43b4af8380d6acae2cf0: Received BeginTabletCopySession request for tablet e710f2a72300483b981e8ca682fdd9c9 from peer bf103d0fdc514c009863847d63610dff ({username='slave'} at 127.0.0.1:52904)
I20251023 02:10:10.969796 20653 tablet_copy_service.cc:161] P 44dc70fd17da43b4af8380d6acae2cf0: Beginning new tablet copy session on tablet e710f2a72300483b981e8ca682fdd9c9 from peer bf103d0fdc514c009863847d63610dff at {username='slave'} at 127.0.0.1:52904: session id = bf103d0fdc514c009863847d63610dff-e710f2a72300483b981e8ca682fdd9c9
I20251023 02:10:10.972222 20653 tablet_copy_source_session.cc:215] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:10.973141 20989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e710f2a72300483b981e8ca682fdd9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:10.978179 20989 tablet_copy_client.cc:806] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:10.978466 20989 tablet_copy_client.cc:670] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:10.979746 20989 tablet_copy_client.cc:538] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:10.982512 20989 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: Bootstrap starting.
I20251023 02:10:10.988459 20989 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:10.988801 20989 tablet_bootstrap.cc:492] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: Bootstrap complete.
I20251023 02:10:10.989043 20989 ts_tablet_manager.cc:1403] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20251023 02:10:10.989799 20989 raft_consensus.cc:359] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } }
I20251023 02:10:10.989949 20989 raft_consensus.cc:740] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: bf103d0fdc514c009863847d63610dff, State: Initialized, Role: LEARNER
I20251023 02:10:10.990166 20989 consensus_queue.cc:260] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: true } }
I20251023 02:10:10.990934 20989 ts_tablet_manager.cc:1434] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20251023 02:10:10.991534 20653 tablet_copy_service.cc:342] P 44dc70fd17da43b4af8380d6acae2cf0: Request end of tablet copy session bf103d0fdc514c009863847d63610dff-e710f2a72300483b981e8ca682fdd9c9 received from {username='slave'} at 127.0.0.1:52904
I20251023 02:10:10.991650 20653 tablet_copy_service.cc:434] P 44dc70fd17da43b4af8380d6acae2cf0: ending tablet copy session bf103d0fdc514c009863847d63610dff-e710f2a72300483b981e8ca682fdd9c9 on tablet e710f2a72300483b981e8ca682fdd9c9 with peer bf103d0fdc514c009863847d63610dff
I20251023 02:10:11.053953 20906 raft_consensus.cc:1217] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251023 02:10:11.179872 20594 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:11.180709 20594 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:11.180871 20594 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:11.218904 20988 raft_consensus.cc:1064] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: attempting to promote NON_VOTER bf103d0fdc514c009863847d63610dff to VOTER
I20251023 02:10:11.219576 20988 consensus_queue.cc:237] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } }
I20251023 02:10:11.221103 20789 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 02:10:11.221103 20906 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 LEARNER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 02:10:11.221534 20717 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 02:10:11.221663 20972 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:11.221994 20978 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:11.222373 20988 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:11.224960 20717 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.225730 20906 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.228521 20972 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.229452 20538 catalog_manager.cc:5649] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd reported cstate change: config changed from index 3 to 4, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "44dc70fd17da43b4af8380d6acae2cf0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.232431 20789 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, bf103d0fdc514c009863847d63610dff (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "67d823b4f54d49d7a5fa5bba42dd61ea" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.244520 20642 consensus_queue.cc:237] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } }
I20251023 02:10:11.245813 20717 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 02:10:11.245846 20906 raft_consensus.cc:1275] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 FOLLOWER]: Refusing update from remote peer 44dc70fd17da43b4af8380d6acae2cf0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 02:10:11.246376 20988 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:10:11.246609 20995 consensus_queue.cc:1048] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:10:11.249038 20717 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.249312 20906 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.252413 20540 catalog_manager.cc:5649] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd reported cstate change: config changed from index 4 to 5, VOTER 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195) evicted. New cstate: current_term: 2 leader_uuid: "44dc70fd17da43b4af8380d6acae2cf0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.271492 20782 tablet_service.cc:1552] Processing DeleteTablet for tablet e710f2a72300483b981e8ca682fdd9c9 with delete_type TABLET_DATA_TOMBSTONED (TS 67d823b4f54d49d7a5fa5bba42dd61ea not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49694
I20251023 02:10:11.272068 20996 tablet_replica.cc:333] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: stopping tablet replica
I20251023 02:10:11.272226 20996 raft_consensus.cc:2243] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:11.272414 20996 raft_consensus.cc:2272] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:11.273056 20996 ts_tablet_manager.cc:1916] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:10:11.276726 20996 ts_tablet_manager.cc:1929] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251023 02:10:11.276842 20996 log.cc:1199] T e710f2a72300483b981e8ca682fdd9c9 P 67d823b4f54d49d7a5fa5bba42dd61ea: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/e710f2a72300483b981e8ca682fdd9c9
I20251023 02:10:11.277431 20526 catalog_manager.cc:4985] TS 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195:43429): tablet e710f2a72300483b981e8ca682fdd9c9 (table test-workload [id=2ec3216251c74595a9ed7e9edd2993c7]) successfully deleted
I20251023 02:10:11.593952 20988 raft_consensus.cc:2955] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 67d823b4f54d49d7a5fa5bba42dd61ea (127.15.161.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "44dc70fd17da43b4af8380d6acae2cf0" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45245 } } peers { permanent_uuid: "867e9ea72c3a4e22abbaaa4cc28e09bd" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 37169 } } peers { permanent_uuid: "bf103d0fdc514c009863847d63610dff" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 41693 } attrs { promote: false } } }
I20251023 02:10:11.596602 20528 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e710f2a72300483b981e8ca682fdd9c9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:13.181277 20594 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:13.181957 20594 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:13.182096 20594 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:15.182423 20594 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:15.183014 20594 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:15.183122 20594 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:17.183394 20594 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:17.184207 20594 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:17.184324 20594 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:17.619150 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:10:17.643908 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:17.644105 16007 tablet_replica.cc:333] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0: stopping tablet replica
I20251023 02:10:17.644256 16007 raft_consensus.cc:2243] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:17.644407 16007 raft_consensus.cc:2272] T a0d08fdc59974dfcb30b8b370a858a54 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.644888 16007 tablet_replica.cc:333] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0: stopping tablet replica
I20251023 02:10:17.645017 16007 raft_consensus.cc:2243] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:10:17.645236 16007 raft_consensus.cc:2272] T e710f2a72300483b981e8ca682fdd9c9 P 44dc70fd17da43b4af8380d6acae2cf0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.661469 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:10:17.664449 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:10:17.671661 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:17.671875 16007 tablet_replica.cc:333] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd: stopping tablet replica
I20251023 02:10:17.672022 16007 raft_consensus.cc:2243] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:17.672143 16007 raft_consensus.cc:2272] T a0d08fdc59974dfcb30b8b370a858a54 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.672539 16007 tablet_replica.cc:333] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd: stopping tablet replica
I20251023 02:10:17.672657 16007 raft_consensus.cc:2243] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:17.672871 16007 raft_consensus.cc:2272] T 1ef0cb65766841289aaff11ae60a23f0 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.673254 16007 tablet_replica.cc:333] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd: stopping tablet replica
I20251023 02:10:17.673382 16007 raft_consensus.cc:2243] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:17.673571 16007 raft_consensus.cc:2272] T e710f2a72300483b981e8ca682fdd9c9 P 867e9ea72c3a4e22abbaaa4cc28e09bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.686971 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:10:17.708272 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:10:17.715554 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:17.715801 16007 tablet_replica.cc:333] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea: stopping tablet replica
I20251023 02:10:17.715943 16007 raft_consensus.cc:2243] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:10:17.716187 16007 raft_consensus.cc:2272] T a0d08fdc59974dfcb30b8b370a858a54 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.716583 16007 tablet_replica.cc:333] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea: stopping tablet replica
I20251023 02:10:17.716670 16007 raft_consensus.cc:2243] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:17.716835 16007 raft_consensus.cc:2272] T 1ef0cb65766841289aaff11ae60a23f0 P 67d823b4f54d49d7a5fa5bba42dd61ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.729924 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:10:17.732622 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:10:17.740274 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:17.740439 16007 tablet_replica.cc:333] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff: stopping tablet replica
I20251023 02:10:17.740573 16007 raft_consensus.cc:2243] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:17.740741 16007 raft_consensus.cc:2272] T e710f2a72300483b981e8ca682fdd9c9 P bf103d0fdc514c009863847d63610dff [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.741113 16007 tablet_replica.cc:333] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff: stopping tablet replica
I20251023 02:10:17.741241 16007 raft_consensus.cc:2243] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:17.741405 16007 raft_consensus.cc:2272] T 1ef0cb65766841289aaff11ae60a23f0 P bf103d0fdc514c009863847d63610dff [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.754186 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:10:17.756702 16007 master.cc:561] Master@127.15.161.254:37883 shutting down...
I20251023 02:10:17.763660 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:17.763803 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:17.763875 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 83954bb958e748f097019be22d8efe07: stopping tablet replica
I20251023 02:10:17.776517 16007 master.cc:583] Master@127.15.161.254:37883 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingUnstableCluster (10670 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor
I20251023 02:10:17.788450 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:36223
I20251023 02:10:17.788836 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:17.790983 20999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.791030 21000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.791211 21002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:17.791352 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:17.791767 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:17.791819 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:17.791841 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185417791837 us; error 0 us; skew 500 ppm
I20251023 02:10:17.791936 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:17.792862 16007 webserver.cc:492] Webserver started at http://127.15.161.254:33757/ using document root <none> and password file <none>
I20251023 02:10:17.793033 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:17.793076 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:17.793159 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:17.793654 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "4d76ec86adac48c887b5634337696500"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:17.795791 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:17.797057 21007 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.797333 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:17.797405 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "4d76ec86adac48c887b5634337696500"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:17.797509 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:17.824496 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:17.824869 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:17.834590 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:36223
I20251023 02:10:17.834607 21058 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:36223 every 8 connection(s)
I20251023 02:10:17.835505 21059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:17.838675 21059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500: Bootstrap starting.
I20251023 02:10:17.839850 21059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:17.841200 21059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500: No bootstrap required, opened a new log
I20251023 02:10:17.841671 21059 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d76ec86adac48c887b5634337696500" member_type: VOTER }
I20251023 02:10:17.841770 21059 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:17.841818 21059 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d76ec86adac48c887b5634337696500, State: Initialized, Role: FOLLOWER
I20251023 02:10:17.841974 21059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d76ec86adac48c887b5634337696500" member_type: VOTER }
I20251023 02:10:17.842061 21059 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:10:17.842113 21059 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:10:17.842218 21059 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:17.843961 21059 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d76ec86adac48c887b5634337696500" member_type: VOTER }
I20251023 02:10:17.844102 21059 leader_election.cc:304] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d76ec86adac48c887b5634337696500; no voters:
I20251023 02:10:17.844391 21059 leader_election.cc:290] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:10:17.844487 21062 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:17.844846 21062 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 1 LEADER]: Becoming Leader. State: Replica: 4d76ec86adac48c887b5634337696500, State: Running, Role: LEADER
I20251023 02:10:17.845042 21062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d76ec86adac48c887b5634337696500" member_type: VOTER }
I20251023 02:10:17.845088 21059 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:10:17.845567 21063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d76ec86adac48c887b5634337696500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d76ec86adac48c887b5634337696500" member_type: VOTER } }
I20251023 02:10:17.845589 21064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d76ec86adac48c887b5634337696500. Latest consensus state: current_term: 1 leader_uuid: "4d76ec86adac48c887b5634337696500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d76ec86adac48c887b5634337696500" member_type: VOTER } }
I20251023 02:10:17.845741 21063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:17.845824 21064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:17.846984 21067 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:10:17.848824 21067 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:10:17.849704 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:10:17.851490 21067 catalog_manager.cc:1357] Generated new cluster ID: 7725650e9ea34d6f99617b16447719bc
I20251023 02:10:17.851565 21067 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:10:17.881896 21067 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:10:17.882501 21067 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:10:17.889101 21067 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500: Generated new TSK 0
I20251023 02:10:17.889268 21067 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:10:17.914614 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:17.916669 21080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.916679 21081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.917276 21083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:17.917414 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:17.917724 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:17.917783 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:17.917809 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185417917807 us; error 0 us; skew 500 ppm
I20251023 02:10:17.917898 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:17.918911 16007 webserver.cc:492] Webserver started at http://127.15.161.193:43343/ using document root <none> and password file <none>
I20251023 02:10:17.919081 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:17.919127 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:17.919210 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:17.919713 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "9ec63335c94f4e2f813810b210f98204"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:17.921802 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:10:17.923077 21088 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.923332 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:17.923403 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "9ec63335c94f4e2f813810b210f98204"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:17.923471 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:17.935951 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:17.936282 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:17.936686 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:17.937326 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:17.937372 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.937414 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:17.937444 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.947659 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:45381
I20251023 02:10:17.947682 21150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:45381 every 8 connection(s)
I20251023 02:10:17.948978 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:17.951555 21155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.951789 21156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.952181 21158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:17.952746 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:17.952958 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:17.953003 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:17.953023 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185417953020 us; error 0 us; skew 500 ppm
I20251023 02:10:17.953136 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:17.954303 16007 webserver.cc:492] Webserver started at http://127.15.161.194:45603/ using document root <none> and password file <none>
I20251023 02:10:17.954496 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:17.954591 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:17.954692 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:17.955250 21151 heartbeater.cc:344] Connected to a master server at 127.15.161.254:36223
I20251023 02:10:17.955271 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "896b31334c514434ab5f6f60ee2d637f"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:17.955356 21151 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:17.955638 21151 heartbeater.cc:507] Master 127.15.161.254:36223 requested a full tablet report, sending...
I20251023 02:10:17.956305 21024 ts_manager.cc:194] Registered new tserver with Master: 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381)
I20251023 02:10:17.957231 21024 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34160
I20251023 02:10:17.957413 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:17.958766 21163 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.959045 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:17.959133 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "896b31334c514434ab5f6f60ee2d637f"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:17.959219 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:17.983314 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:17.983659 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:17.984048 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:17.984640 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:17.984688 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.984745 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:17.984781 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:17.995755 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:41603
I20251023 02:10:17.995784 21225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:41603 every 8 connection(s)
I20251023 02:10:17.996958 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:17.999305 21229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:17.999359 21230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:18.000013 21232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:18.001065 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:18.001323 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:18.001372 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:18.001401 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185418001397 us; error 0 us; skew 500 ppm
I20251023 02:10:18.001528 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:18.002111 21226 heartbeater.cc:344] Connected to a master server at 127.15.161.254:36223
I20251023 02:10:18.002221 21226 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:18.002514 21226 heartbeater.cc:507] Master 127.15.161.254:36223 requested a full tablet report, sending...
I20251023 02:10:18.002568 16007 webserver.cc:492] Webserver started at http://127.15.161.195:34757/ using document root <none> and password file <none>
I20251023 02:10:18.002730 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:18.002789 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:18.002875 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:18.003170 21024 ts_manager.cc:194] Registered new tserver with Master: 896b31334c514434ab5f6f60ee2d637f (127.15.161.194:41603)
I20251023 02:10:18.003365 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "5799855af2fd4ce68c3819d832a6d4a7"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:18.003756 21024 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34172
I20251023 02:10:18.005414 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:18.006625 21237 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:18.006911 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:18.006981 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "5799855af2fd4ce68c3819d832a6d4a7"
format_stamp: "Formatted at 2025-10-23 02:10:17 on dist-test-slave-bfbt"
I20251023 02:10:18.007045 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:18.024761 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:18.025067 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:18.025416 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:18.025988 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:18.026031 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:18.026075 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:18.026100 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:18.036166 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:42839
I20251023 02:10:18.036185 21299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:42839 every 8 connection(s)
I20251023 02:10:18.041497 21300 heartbeater.cc:344] Connected to a master server at 127.15.161.254:36223
I20251023 02:10:18.041584 21300 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:18.041790 21300 heartbeater.cc:507] Master 127.15.161.254:36223 requested a full tablet report, sending...
I20251023 02:10:18.042404 21024 ts_manager.cc:194] Registered new tserver with Master: 5799855af2fd4ce68c3819d832a6d4a7 (127.15.161.195:42839)
I20251023 02:10:18.042574 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005599035s
I20251023 02:10:18.042971 21024 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34178
I20251023 02:10:18.958441 21151 heartbeater.cc:499] Master 127.15.161.254:36223 was elected leader, sending a full tablet report...
I20251023 02:10:19.004877 21226 heartbeater.cc:499] Master 127.15.161.254:36223 was elected leader, sending a full tablet report...
I20251023 02:10:19.043865 21300 heartbeater.cc:499] Master 127.15.161.254:36223 was elected leader, sending a full tablet report...
I20251023 02:10:19.069419 16007 test_util.cc:276] Using random seed: -280393752
I20251023 02:10:19.082160 21024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34188:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:10:19.082590 21024 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:10:19.095362 21116 tablet_service.cc:1505] Processing CreateTablet for tablet 3946f5df8a964904bd542e4690ab0786 (DEFAULT_TABLE table=test-workload [id=58e32b5dc09347f8bddaf11ec25220eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:10:19.095745 21116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3946f5df8a964904bd542e4690ab0786. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:19.095829 21191 tablet_service.cc:1505] Processing CreateTablet for tablet 3946f5df8a964904bd542e4690ab0786 (DEFAULT_TABLE table=test-workload [id=58e32b5dc09347f8bddaf11ec25220eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:10:19.096132 21191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3946f5df8a964904bd542e4690ab0786. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:19.096112 21265 tablet_service.cc:1505] Processing CreateTablet for tablet 3946f5df8a964904bd542e4690ab0786 (DEFAULT_TABLE table=test-workload [id=58e32b5dc09347f8bddaf11ec25220eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 02:10:19.096391 21265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3946f5df8a964904bd542e4690ab0786. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:19.100193 21320 tablet_bootstrap.cc:492] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f: Bootstrap starting.
I20251023 02:10:19.101435 21320 tablet_bootstrap.cc:654] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:19.106549 21322 tablet_bootstrap.cc:492] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7: Bootstrap starting.
I20251023 02:10:19.107100 21321 tablet_bootstrap.cc:492] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204: Bootstrap starting.
I20251023 02:10:19.108513 21320 tablet_bootstrap.cc:492] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f: No bootstrap required, opened a new log
I20251023 02:10:19.108631 21320 ts_tablet_manager.cc:1403] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20251023 02:10:19.108754 21322 tablet_bootstrap.cc:654] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:19.109117 21321 tablet_bootstrap.cc:654] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:19.109413 21320 raft_consensus.cc:359] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.109637 21320 raft_consensus.cc:385] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:19.109748 21320 raft_consensus.cc:740] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896b31334c514434ab5f6f60ee2d637f, State: Initialized, Role: FOLLOWER
I20251023 02:10:19.109964 21320 consensus_queue.cc:260] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.110507 21322 tablet_bootstrap.cc:492] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7: No bootstrap required, opened a new log
I20251023 02:10:19.110651 21322 ts_tablet_manager.cc:1403] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:19.110703 21320 ts_tablet_manager.cc:1434] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:19.110805 21321 tablet_bootstrap.cc:492] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204: No bootstrap required, opened a new log
I20251023 02:10:19.110945 21321 ts_tablet_manager.cc:1403] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:19.111532 21321 raft_consensus.cc:359] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.111553 21322 raft_consensus.cc:359] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.111694 21321 raft_consensus.cc:385] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:19.111718 21322 raft_consensus.cc:385] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:19.111783 21321 raft_consensus.cc:740] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ec63335c94f4e2f813810b210f98204, State: Initialized, Role: FOLLOWER
I20251023 02:10:19.111850 21322 raft_consensus.cc:740] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5799855af2fd4ce68c3819d832a6d4a7, State: Initialized, Role: FOLLOWER
I20251023 02:10:19.112071 21322 consensus_queue.cc:260] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.112071 21321 consensus_queue.cc:260] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.112788 21322 ts_tablet_manager.cc:1434] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:19.114004 21321 ts_tablet_manager.cc:1434] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251023 02:10:19.130273 21327 raft_consensus.cc:493] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:19.130388 21327 raft_consensus.cc:515] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.130926 21327 leader_election.cc:290] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381), 896b31334c514434ab5f6f60ee2d637f (127.15.161.194:41603)
I20251023 02:10:19.136583 21126 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3946f5df8a964904bd542e4690ab0786" candidate_uuid: "5799855af2fd4ce68c3819d832a6d4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ec63335c94f4e2f813810b210f98204" is_pre_election: true
I20251023 02:10:19.136585 21201 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3946f5df8a964904bd542e4690ab0786" candidate_uuid: "5799855af2fd4ce68c3819d832a6d4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896b31334c514434ab5f6f60ee2d637f" is_pre_election: true
I20251023 02:10:19.136736 21126 raft_consensus.cc:2468] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5799855af2fd4ce68c3819d832a6d4a7 in term 0.
I20251023 02:10:19.136742 21201 raft_consensus.cc:2468] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5799855af2fd4ce68c3819d832a6d4a7 in term 0.
I20251023 02:10:19.137069 21239 leader_election.cc:304] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5799855af2fd4ce68c3819d832a6d4a7, 896b31334c514434ab5f6f60ee2d637f, 9ec63335c94f4e2f813810b210f98204; no voters:
I20251023 02:10:19.137251 21327 raft_consensus.cc:2804] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:19.137318 21327 raft_consensus.cc:493] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:19.137379 21327 raft_consensus.cc:3060] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:19.139043 21327 raft_consensus.cc:515] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.139477 21327 leader_election.cc:290] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [CANDIDATE]: Term 1 election: Requested vote from peers 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381), 896b31334c514434ab5f6f60ee2d637f (127.15.161.194:41603)
I20251023 02:10:19.139812 21201 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3946f5df8a964904bd542e4690ab0786" candidate_uuid: "5799855af2fd4ce68c3819d832a6d4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896b31334c514434ab5f6f60ee2d637f"
I20251023 02:10:19.139789 21126 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3946f5df8a964904bd542e4690ab0786" candidate_uuid: "5799855af2fd4ce68c3819d832a6d4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ec63335c94f4e2f813810b210f98204"
I20251023 02:10:19.139956 21126 raft_consensus.cc:3060] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:19.139954 21201 raft_consensus.cc:3060] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:19.142110 21126 raft_consensus.cc:2468] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5799855af2fd4ce68c3819d832a6d4a7 in term 1.
I20251023 02:10:19.142110 21201 raft_consensus.cc:2468] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5799855af2fd4ce68c3819d832a6d4a7 in term 1.
I20251023 02:10:19.142475 21240 leader_election.cc:304] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5799855af2fd4ce68c3819d832a6d4a7, 896b31334c514434ab5f6f60ee2d637f, 9ec63335c94f4e2f813810b210f98204; no voters:
I20251023 02:10:19.142685 21327 raft_consensus.cc:2804] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:19.142976 21327 raft_consensus.cc:697] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 1 LEADER]: Becoming Leader. State: Replica: 5799855af2fd4ce68c3819d832a6d4a7, State: Running, Role: LEADER
I20251023 02:10:19.143146 21327 consensus_queue.cc:237] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } }
I20251023 02:10:19.144686 21024 catalog_manager.cc:5649] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5799855af2fd4ce68c3819d832a6d4a7 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "5799855af2fd4ce68c3819d832a6d4a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ec63335c94f4e2f813810b210f98204" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 45381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5799855af2fd4ce68c3819d832a6d4a7" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 42839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:19.151150 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:10:19.157408 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:19.157660 16007 tablet_replica.cc:333] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204: stopping tablet replica
I20251023 02:10:19.157824 16007 raft_consensus.cc:2243] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:19.157939 16007 raft_consensus.cc:2272] T 3946f5df8a964904bd542e4690ab0786 P 9ec63335c94f4e2f813810b210f98204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:19.171123 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
W20251023 02:10:19.643447 21239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20251023 02:10:19.644399 21239 consensus_peers.cc:597] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 -> Peer 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381): Couldn't send request to peer 9ec63335c94f4e2f813810b210f98204. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:19.658972 21327 consensus_queue.cc:1048] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "896b31334c514434ab5f6f60ee2d637f" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 41603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:19.849617 21077 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:19.849996 21077 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:19.850080 21077 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:21.850521 21077 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:21.850925 21077 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:21.851004 21077 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 02:10:22.213750 21239 consensus_peers.cc:597] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 -> Peer 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381): Couldn't send request to peer 9ec63335c94f4e2f813810b210f98204. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20251023 02:10:23.851415 21077 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:23.851823 21077 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:23.851895 21077 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 02:10:24.951264 21239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20251023 02:10:24.952157 21239 consensus_peers.cc:597] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 -> Peer 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381): Couldn't send request to peer 9ec63335c94f4e2f813810b210f98204. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251023 02:10:25.852296 21077 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:25.852725 21077 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:25.852813 21077 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 02:10:27.311537 21239 consensus_peers.cc:597] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 -> Peer 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381): Couldn't send request to peer 9ec63335c94f4e2f813810b210f98204. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251023 02:10:27.853219 21077 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:27.853605 21077 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:27.853677 21077 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 02:10:29.853679 21239 consensus_peers.cc:597] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 -> Peer 9ec63335c94f4e2f813810b210f98204 (127.15.161.193:45381): Couldn't send request to peer 9ec63335c94f4e2f813810b210f98204. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:45381: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20251023 02:10:29.853958 21077 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 02:10:29.854306 21077 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 02:10:29.854386 21077 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 02:10:30.234170 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:10:30.252964 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:30.253232 16007 tablet_replica.cc:333] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f: stopping tablet replica
I20251023 02:10:30.253396 16007 raft_consensus.cc:2243] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:30.253553 16007 raft_consensus.cc:2272] T 3946f5df8a964904bd542e4690ab0786 P 896b31334c514434ab5f6f60ee2d637f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:30.269820 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:10:30.272447 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:10:30.279626 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:30.279820 16007 tablet_replica.cc:333] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7: stopping tablet replica
I20251023 02:10:30.279958 16007 raft_consensus.cc:2243] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:30.280236 16007 raft_consensus.cc:2272] T 3946f5df8a964904bd542e4690ab0786 P 5799855af2fd4ce68c3819d832a6d4a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:30.306222 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:10:30.308619 16007 master.cc:561] Master@127.15.161.254:36223 shutting down...
I20251023 02:10:30.315641 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:30.315810 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:30.315894 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4d76ec86adac48c887b5634337696500: stopping tablet replica
I20251023 02:10:30.328341 16007 master.cc:583] Master@127.15.161.254:36223 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor (12549 ms)
[ RUN ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering
I20251023 02:10:30.337935 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:41493
I20251023 02:10:30.338346 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:30.340345 21364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.340413 21365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.340513 21367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:30.340934 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:30.341156 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:30.341203 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:30.341240 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185430341236 us; error 0 us; skew 500 ppm
I20251023 02:10:30.341395 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:30.342350 16007 webserver.cc:492] Webserver started at http://127.15.161.254:39543/ using document root <none> and password file <none>
I20251023 02:10:30.342532 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:30.342597 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:30.342707 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:30.343246 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "939799739d5e4f6fb69056e4a5629024"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.345307 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:30.346549 21372 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.346809 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:30.346889 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "939799739d5e4f6fb69056e4a5629024"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.346972 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:30.361917 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:30.362288 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:30.371727 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:41493
I20251023 02:10:30.371744 21423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:41493 every 8 connection(s)
I20251023 02:10:30.372646 21424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:30.375958 21424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024: Bootstrap starting.
I20251023 02:10:30.377075 21424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:30.378360 21424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024: No bootstrap required, opened a new log
I20251023 02:10:30.378935 21424 raft_consensus.cc:359] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939799739d5e4f6fb69056e4a5629024" member_type: VOTER }
I20251023 02:10:30.379062 21424 raft_consensus.cc:385] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:30.379103 21424 raft_consensus.cc:740] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939799739d5e4f6fb69056e4a5629024, State: Initialized, Role: FOLLOWER
I20251023 02:10:30.379264 21424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939799739d5e4f6fb69056e4a5629024" member_type: VOTER }
I20251023 02:10:30.379392 21424 raft_consensus.cc:399] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:10:30.379482 21424 raft_consensus.cc:493] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:10:30.379554 21424 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:30.381695 21424 raft_consensus.cc:515] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939799739d5e4f6fb69056e4a5629024" member_type: VOTER }
I20251023 02:10:30.381883 21424 leader_election.cc:304] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 939799739d5e4f6fb69056e4a5629024; no voters:
I20251023 02:10:30.387538 21424 leader_election.cc:290] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:10:30.387615 21427 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:30.388269 21424 sys_catalog.cc:565] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:10:30.389566 21427 raft_consensus.cc:697] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 1 LEADER]: Becoming Leader. State: Replica: 939799739d5e4f6fb69056e4a5629024, State: Running, Role: LEADER
I20251023 02:10:30.389806 21427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939799739d5e4f6fb69056e4a5629024" member_type: VOTER }
I20251023 02:10:30.390555 21429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 939799739d5e4f6fb69056e4a5629024. Latest consensus state: current_term: 1 leader_uuid: "939799739d5e4f6fb69056e4a5629024" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939799739d5e4f6fb69056e4a5629024" member_type: VOTER } }
I20251023 02:10:30.390693 21429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:30.390898 21428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "939799739d5e4f6fb69056e4a5629024" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939799739d5e4f6fb69056e4a5629024" member_type: VOTER } }
I20251023 02:10:30.391036 21428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:30.392901 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251023 02:10:30.393512 21443 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 02:10:30.393573 21443 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 02:10:30.393635 21436 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:10:30.394574 21436 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:10:30.396658 21436 catalog_manager.cc:1357] Generated new cluster ID: fc6555bdf54546e4afb5a7fcd9400211
I20251023 02:10:30.396714 21436 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:10:30.416070 21436 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:10:30.416708 21436 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:10:30.430018 21436 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024: Generated new TSK 0
I20251023 02:10:30.430212 21436 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:10:30.457404 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:30.459602 21445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.459669 21446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.460168 21448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:30.460716 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:30.460952 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:30.461001 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:30.461031 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185430461029 us; error 0 us; skew 500 ppm
I20251023 02:10:30.461130 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:30.462057 16007 webserver.cc:492] Webserver started at http://127.15.161.193:40195/ using document root <none> and password file <none>
I20251023 02:10:30.462272 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:30.462321 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:30.462402 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:30.462852 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "f82073061a834a5a829f914b1f9071b3"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.464779 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:30.465989 21453 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.466320 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:30.466396 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "f82073061a834a5a829f914b1f9071b3"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.466490 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:30.488559 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:30.488926 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:30.489331 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:30.490005 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:30.490051 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.490108 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:30.490162 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.499912 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:34467
I20251023 02:10:30.499936 21515 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:34467 every 8 connection(s)
I20251023 02:10:30.501264 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:30.503281 21520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.504241 21521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.505429 21523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:30.505523 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:30.505882 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:30.505949 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:30.505985 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185430505981 us; error 0 us; skew 500 ppm
I20251023 02:10:30.506110 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:30.507129 16007 webserver.cc:492] Webserver started at http://127.15.161.194:33903/ using document root <none> and password file <none>
I20251023 02:10:30.507130 21516 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41493
I20251023 02:10:30.507275 21516 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:30.507340 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:30.507402 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:30.507512 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:30.507577 21516 heartbeater.cc:507] Master 127.15.161.254:41493 requested a full tablet report, sending...
I20251023 02:10:30.508018 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.508312 21389 ts_manager.cc:194] Registered new tserver with Master: f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467)
I20251023 02:10:30.509097 21389 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46366
I20251023 02:10:30.509977 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:30.511408 21528 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.511726 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:30.511796 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.511860 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:30.516288 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:30.516683 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:30.517069 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:30.517637 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:30.517680 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.517732 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:30.517767 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.527413 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:38737
I20251023 02:10:30.527439 21590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:38737 every 8 connection(s)
I20251023 02:10:30.528566 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:30.530928 21594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.531085 21595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:30.533195 21597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:30.533500 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:30.533687 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:30.533732 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:30.533762 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185430533758 us; error 0 us; skew 500 ppm
I20251023 02:10:30.533869 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:30.533906 21591 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41493
I20251023 02:10:30.533990 21591 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:30.534271 21591 heartbeater.cc:507] Master 127.15.161.254:41493 requested a full tablet report, sending...
I20251023 02:10:30.534868 16007 webserver.cc:492] Webserver started at http://127.15.161.195:33195/ using document root <none> and password file <none>
I20251023 02:10:30.534996 21389 ts_manager.cc:194] Registered new tserver with Master: ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:30.535068 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:30.535122 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:30.535205 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:30.535738 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.535790 21389 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46370
I20251023 02:10:30.537765 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:10:30.538945 21602 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.539216 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:30.539285 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db"
format_stamp: "Formatted at 2025-10-23 02:10:30 on dist-test-slave-bfbt"
I20251023 02:10:30.539364 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:30.553045 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:30.553395 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:30.553797 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:30.554418 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:30.554462 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.554508 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:30.554548 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:30.564388 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:45609
I20251023 02:10:30.564409 21664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:45609 every 8 connection(s)
I20251023 02:10:30.569792 21665 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41493
I20251023 02:10:30.569878 21665 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:30.570108 21665 heartbeater.cc:507] Master 127.15.161.254:41493 requested a full tablet report, sending...
I20251023 02:10:30.570854 21389 ts_manager.cc:194] Registered new tserver with Master: 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609)
I20251023 02:10:30.571435 21389 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46384
I20251023 02:10:30.571763 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00669121s
I20251023 02:10:31.510154 21516 heartbeater.cc:499] Master 127.15.161.254:41493 was elected leader, sending a full tablet report...
I20251023 02:10:31.536965 21591 heartbeater.cc:499] Master 127.15.161.254:41493 was elected leader, sending a full tablet report...
I20251023 02:10:31.572467 21665 heartbeater.cc:499] Master 127.15.161.254:41493 was elected leader, sending a full tablet report...
I20251023 02:10:31.598865 16007 test_util.cc:276] Using random seed: -267864307
I20251023 02:10:31.609136 21389 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46394:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:10:31.609648 21389 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:10:31.632946 21630 tablet_service.cc:1505] Processing CreateTablet for tablet d8b104894f3b48e48218fd262ada68c8 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:31.633031 21629 tablet_service.cc:1505] Processing CreateTablet for tablet 636dcc63792e431591309553dd780724 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:31.633358 21630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8b104894f3b48e48218fd262ada68c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.633605 21628 tablet_service.cc:1505] Processing CreateTablet for tablet 92c1cc41f8a34158ab49084fc756f5f1 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:31.634196 21627 tablet_service.cc:1505] Processing CreateTablet for tablet 5f73f4ecf0114cf19b74ecdff72bad50 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:31.634902 21556 tablet_service.cc:1505] Processing CreateTablet for tablet d8b104894f3b48e48218fd262ada68c8 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:31.635051 21555 tablet_service.cc:1505] Processing CreateTablet for tablet 636dcc63792e431591309553dd780724 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:31.635222 21556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8b104894f3b48e48218fd262ada68c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.635383 21555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636dcc63792e431591309553dd780724. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.637519 21554 tablet_service.cc:1505] Processing CreateTablet for tablet 92c1cc41f8a34158ab49084fc756f5f1 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:31.637814 21554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c1cc41f8a34158ab49084fc756f5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.638541 21628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c1cc41f8a34158ab49084fc756f5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.642407 21627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f73f4ecf0114cf19b74ecdff72bad50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.642686 21555 tablet_service.cc:1505] Processing CreateTablet for tablet 5f73f4ecf0114cf19b74ecdff72bad50 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:31.642956 21555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f73f4ecf0114cf19b74ecdff72bad50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.646466 21629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636dcc63792e431591309553dd780724. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.650640 21686 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Bootstrap starting.
I20251023 02:10:31.650934 21481 tablet_service.cc:1505] Processing CreateTablet for tablet d8b104894f3b48e48218fd262ada68c8 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:31.651041 21480 tablet_service.cc:1505] Processing CreateTablet for tablet 636dcc63792e431591309553dd780724 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:31.651333 21481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8b104894f3b48e48218fd262ada68c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.651283 21478 tablet_service.cc:1505] Processing CreateTablet for tablet 5f73f4ecf0114cf19b74ecdff72bad50 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:31.651621 21478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f73f4ecf0114cf19b74ecdff72bad50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.651708 21685 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: Bootstrap starting.
I20251023 02:10:31.651882 21479 tablet_service.cc:1505] Processing CreateTablet for tablet 92c1cc41f8a34158ab49084fc756f5f1 (DEFAULT_TABLE table=test-workload [id=a64064023f9a49908ca9c3400601aa71]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:31.652231 21479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c1cc41f8a34158ab49084fc756f5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.652402 21480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636dcc63792e431591309553dd780724. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:31.653699 21685 tablet_bootstrap.cc:654] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.659126 21686 tablet_bootstrap.cc:654] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.665294 21686 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: No bootstrap required, opened a new log
I20251023 02:10:31.665413 21686 ts_tablet_manager.cc:1403] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20251023 02:10:31.666180 21686 raft_consensus.cc:359] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.666321 21686 raft_consensus.cc:385] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.666394 21686 raft_consensus.cc:740] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6f3b5fe80b4fae8ebe4328a8fb67db, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.666610 21686 consensus_queue.cc:260] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.667584 21686 ts_tablet_manager.cc:1434] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:31.667979 21688 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3: Bootstrap starting.
I20251023 02:10:31.668342 21686 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Bootstrap starting.
I20251023 02:10:31.668403 21685 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: No bootstrap required, opened a new log
I20251023 02:10:31.668579 21685 ts_tablet_manager.cc:1403] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.013s
I20251023 02:10:31.669334 21685 raft_consensus.cc:359] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.669523 21685 raft_consensus.cc:385] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.669610 21685 raft_consensus.cc:740] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfdeb364bed4ee5acbd9f5d365e0aef, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.669634 21688 tablet_bootstrap.cc:654] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.669826 21685 consensus_queue.cc:260] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.670490 21685 ts_tablet_manager.cc:1434] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:31.670904 21685 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: Bootstrap starting.
I20251023 02:10:31.672139 21685 tablet_bootstrap.cc:654] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.672586 21686 tablet_bootstrap.cc:654] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.675851 21686 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: No bootstrap required, opened a new log
I20251023 02:10:31.675987 21686 ts_tablet_manager.cc:1403] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251023 02:10:31.676802 21686 raft_consensus.cc:359] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.676961 21686 raft_consensus.cc:385] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.677017 21686 raft_consensus.cc:740] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6f3b5fe80b4fae8ebe4328a8fb67db, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.677182 21686 consensus_queue.cc:260] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.677456 21688 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3: No bootstrap required, opened a new log
I20251023 02:10:31.677482 21686 ts_tablet_manager.cc:1434] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:31.677582 21688 ts_tablet_manager.cc:1403] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20251023 02:10:31.677754 21686 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Bootstrap starting.
I20251023 02:10:31.678179 21688 raft_consensus.cc:359] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.678411 21688 raft_consensus.cc:385] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.678514 21688 raft_consensus.cc:740] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f82073061a834a5a829f914b1f9071b3, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.678768 21688 consensus_queue.cc:260] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.678979 21685 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: No bootstrap required, opened a new log
I20251023 02:10:31.679026 21686 tablet_bootstrap.cc:654] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.679131 21685 ts_tablet_manager.cc:1403] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20251023 02:10:31.679793 21685 raft_consensus.cc:359] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.680029 21685 raft_consensus.cc:385] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.680128 21685 raft_consensus.cc:740] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfdeb364bed4ee5acbd9f5d365e0aef, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.680467 21685 consensus_queue.cc:260] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.681210 21685 ts_tablet_manager.cc:1434] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:31.681574 21685 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef: Bootstrap starting.
I20251023 02:10:31.683390 21685 tablet_bootstrap.cc:654] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.686888 21688 ts_tablet_manager.cc:1434] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20251023 02:10:31.687173 21688 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3: Bootstrap starting.
I20251023 02:10:31.687511 21686 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: No bootstrap required, opened a new log
I20251023 02:10:31.687558 21685 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef: No bootstrap required, opened a new log
I20251023 02:10:31.687682 21686 ts_tablet_manager.cc:1403] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20251023 02:10:31.687716 21685 ts_tablet_manager.cc:1403] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20251023 02:10:31.688323 21686 raft_consensus.cc:359] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.688449 21686 raft_consensus.cc:385] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.688422 21685 raft_consensus.cc:359] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.688530 21686 raft_consensus.cc:740] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6f3b5fe80b4fae8ebe4328a8fb67db, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.688592 21685 raft_consensus.cc:385] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.688697 21685 raft_consensus.cc:740] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfdeb364bed4ee5acbd9f5d365e0aef, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.688777 21686 consensus_queue.cc:260] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.688966 21685 consensus_queue.cc:260] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.689234 21686 ts_tablet_manager.cc:1434] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:31.689384 21685 ts_tablet_manager.cc:1434] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:31.689508 21686 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Bootstrap starting.
I20251023 02:10:31.689694 21685 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef: Bootstrap starting.
I20251023 02:10:31.689890 21688 tablet_bootstrap.cc:654] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.691154 21686 tablet_bootstrap.cc:654] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.691661 21685 tablet_bootstrap.cc:654] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.698038 21688 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3: No bootstrap required, opened a new log
I20251023 02:10:31.698244 21688 ts_tablet_manager.cc:1403] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20251023 02:10:31.698519 21686 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: No bootstrap required, opened a new log
I20251023 02:10:31.698642 21686 ts_tablet_manager.cc:1403] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20251023 02:10:31.698835 21685 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef: No bootstrap required, opened a new log
I20251023 02:10:31.698999 21685 ts_tablet_manager.cc:1403] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20251023 02:10:31.698937 21688 raft_consensus.cc:359] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.699174 21688 raft_consensus.cc:385] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.699280 21688 raft_consensus.cc:740] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f82073061a834a5a829f914b1f9071b3, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.699280 21686 raft_consensus.cc:359] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.699509 21686 raft_consensus.cc:385] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.699641 21686 raft_consensus.cc:740] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6f3b5fe80b4fae8ebe4328a8fb67db, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.699574 21688 consensus_queue.cc:260] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.699771 21685 raft_consensus.cc:359] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.699893 21686 consensus_queue.cc:260] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.700021 21688 ts_tablet_manager.cc:1434] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:31.700069 21685 raft_consensus.cc:385] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.700673 21685 raft_consensus.cc:740] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfdeb364bed4ee5acbd9f5d365e0aef, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.700919 21685 consensus_queue.cc:260] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.701308 21685 ts_tablet_manager.cc:1434] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:31.700304 21688 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3: Bootstrap starting.
I20251023 02:10:31.703433 21688 tablet_bootstrap.cc:654] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.700434 21686 ts_tablet_manager.cc:1434] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:31.705106 21688 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3: No bootstrap required, opened a new log
I20251023 02:10:31.705230 21688 ts_tablet_manager.cc:1403] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251023 02:10:31.705806 21688 raft_consensus.cc:359] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.705915 21688 raft_consensus.cc:385] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.705968 21688 raft_consensus.cc:740] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f82073061a834a5a829f914b1f9071b3, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.706110 21688 consensus_queue.cc:260] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.706459 21688 ts_tablet_manager.cc:1434] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:31.706719 21688 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3: Bootstrap starting.
I20251023 02:10:31.708002 21688 tablet_bootstrap.cc:654] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:31.709429 21688 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3: No bootstrap required, opened a new log
I20251023 02:10:31.709566 21688 ts_tablet_manager.cc:1403] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20251023 02:10:31.709996 21688 raft_consensus.cc:359] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.710103 21688 raft_consensus.cc:385] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:31.710184 21688 raft_consensus.cc:740] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f82073061a834a5a829f914b1f9071b3, State: Initialized, Role: FOLLOWER
I20251023 02:10:31.710379 21688 consensus_queue.cc:260] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.710739 21688 ts_tablet_manager.cc:1434] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:31.714078 21691 raft_consensus.cc:493] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:31.714203 21691 raft_consensus.cc:515] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.714754 21691 leader_election.cc:290] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609), f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467)
I20251023 02:10:31.720642 21640 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8b104894f3b48e48218fd262ada68c8" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" is_pre_election: true
I20251023 02:10:31.720835 21640 raft_consensus.cc:2468] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 0.
I20251023 02:10:31.721246 21531 leader_election.cc:304] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, ddfdeb364bed4ee5acbd9f5d365e0aef; no voters:
I20251023 02:10:31.721509 21691 raft_consensus.cc:2804] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:31.721632 21691 raft_consensus.cc:493] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:31.721721 21691 raft_consensus.cc:3060] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.723383 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8b104894f3b48e48218fd262ada68c8" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3" is_pre_election: true
I20251023 02:10:31.723492 21691 raft_consensus.cc:515] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.723543 21491 raft_consensus.cc:2468] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 0.
I20251023 02:10:31.723946 21691 leader_election.cc:290] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 election: Requested vote from peers 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609), f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467)
I20251023 02:10:31.724357 21640 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8b104894f3b48e48218fd262ada68c8" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db"
I20251023 02:10:31.724381 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d8b104894f3b48e48218fd262ada68c8" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3"
I20251023 02:10:31.724510 21491 raft_consensus.cc:3060] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.724514 21640 raft_consensus.cc:3060] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.726370 21491 raft_consensus.cc:2468] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 1.
I20251023 02:10:31.726554 21640 raft_consensus.cc:2468] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 1.
I20251023 02:10:31.726665 21531 leader_election.cc:304] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ddfdeb364bed4ee5acbd9f5d365e0aef, f82073061a834a5a829f914b1f9071b3; no voters:
I20251023 02:10:31.726900 21691 raft_consensus.cc:2804] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:31.727203 21691 raft_consensus.cc:697] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Becoming Leader. State: Replica: ddfdeb364bed4ee5acbd9f5d365e0aef, State: Running, Role: LEADER
I20251023 02:10:31.727422 21691 consensus_queue.cc:237] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } }
I20251023 02:10:31.729035 21388 catalog_manager.cc:5649] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef reported cstate change: term changed from 0 to 1, leader changed from <none> to ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194). New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:31.734467 21690 raft_consensus.cc:493] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:31.734581 21690 raft_consensus.cc:515] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.735129 21690 leader_election.cc:290] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467), ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:31.740695 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5f73f4ecf0114cf19b74ecdff72bad50" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3" is_pre_election: true
I20251023 02:10:31.740768 21566 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5f73f4ecf0114cf19b74ecdff72bad50" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" is_pre_election: true
I20251023 02:10:31.740906 21491 raft_consensus.cc:2468] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 0.
I20251023 02:10:31.740952 21566 raft_consensus.cc:2468] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 0.
I20251023 02:10:31.741256 21605 leader_election.cc:304] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, f82073061a834a5a829f914b1f9071b3; no voters:
I20251023 02:10:31.741500 21690 raft_consensus.cc:2804] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:31.741576 21690 raft_consensus.cc:493] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:31.741655 21690 raft_consensus.cc:3060] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.743208 21690 raft_consensus.cc:515] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.743588 21690 leader_election.cc:290] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 election: Requested vote from peers f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467), ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:31.743896 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5f73f4ecf0114cf19b74ecdff72bad50" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3"
I20251023 02:10:31.743984 21566 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5f73f4ecf0114cf19b74ecdff72bad50" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef"
I20251023 02:10:31.744035 21491 raft_consensus.cc:3060] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.744145 21566 raft_consensus.cc:3060] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.745642 21491 raft_consensus.cc:2468] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 1.
I20251023 02:10:31.745733 21566 raft_consensus.cc:2468] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 1.
I20251023 02:10:31.745944 21605 leader_election.cc:304] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, f82073061a834a5a829f914b1f9071b3; no voters:
I20251023 02:10:31.746161 21690 raft_consensus.cc:2804] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:31.746466 21690 raft_consensus.cc:697] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Becoming Leader. State: Replica: 0b6f3b5fe80b4fae8ebe4328a8fb67db, State: Running, Role: LEADER
I20251023 02:10:31.746656 21690 consensus_queue.cc:237] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.748299 21388 catalog_manager.cc:5649] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195). New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:31.781136 21690 raft_consensus.cc:493] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:31.781272 21690 raft_consensus.cc:515] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.781708 21690 leader_election.cc:290] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467), ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:31.782027 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92c1cc41f8a34158ab49084fc756f5f1" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3" is_pre_election: true
I20251023 02:10:31.782076 21566 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92c1cc41f8a34158ab49084fc756f5f1" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" is_pre_election: true
I20251023 02:10:31.782191 21491 raft_consensus.cc:2468] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 0.
I20251023 02:10:31.782253 21566 raft_consensus.cc:2468] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 0.
I20251023 02:10:31.782505 21605 leader_election.cc:304] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, f82073061a834a5a829f914b1f9071b3; no voters:
I20251023 02:10:31.782711 21690 raft_consensus.cc:2804] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:31.782786 21690 raft_consensus.cc:493] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:31.782866 21690 raft_consensus.cc:3060] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.784632 21690 raft_consensus.cc:515] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.785125 21690 leader_election.cc:290] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 election: Requested vote from peers f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467), ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:31.785446 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92c1cc41f8a34158ab49084fc756f5f1" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3"
I20251023 02:10:31.785504 21566 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92c1cc41f8a34158ab49084fc756f5f1" candidate_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef"
I20251023 02:10:31.785593 21491 raft_consensus.cc:3060] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.785672 21566 raft_consensus.cc:3060] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.787278 21491 raft_consensus.cc:2468] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 1.
I20251023 02:10:31.787282 21566 raft_consensus.cc:2468] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b6f3b5fe80b4fae8ebe4328a8fb67db in term 1.
I20251023 02:10:31.787556 21605 leader_election.cc:304] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, f82073061a834a5a829f914b1f9071b3; no voters:
I20251023 02:10:31.787744 21690 raft_consensus.cc:2804] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:31.787889 21690 raft_consensus.cc:697] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Becoming Leader. State: Replica: 0b6f3b5fe80b4fae8ebe4328a8fb67db, State: Running, Role: LEADER
I20251023 02:10:31.788118 21690 consensus_queue.cc:237] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.790053 21388 catalog_manager.cc:5649] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195). New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:31.816193 21691 raft_consensus.cc:493] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:31.816339 21691 raft_consensus.cc:515] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.816794 21691 leader_election.cc:290] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467), 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609)
I20251023 02:10:31.817121 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "636dcc63792e431591309553dd780724" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3" is_pre_election: true
I20251023 02:10:31.817207 21640 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "636dcc63792e431591309553dd780724" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" is_pre_election: true
I20251023 02:10:31.817299 21491 raft_consensus.cc:2468] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 0.
I20251023 02:10:31.817344 21640 raft_consensus.cc:2468] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 0.
I20251023 02:10:31.817600 21531 leader_election.cc:304] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, ddfdeb364bed4ee5acbd9f5d365e0aef; no voters:
I20251023 02:10:31.817786 21691 raft_consensus.cc:2804] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:31.817857 21691 raft_consensus.cc:493] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:31.817914 21691 raft_consensus.cc:3060] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.819449 21691 raft_consensus.cc:515] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.819871 21691 leader_election.cc:290] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 election: Requested vote from peers f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467), 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609)
I20251023 02:10:31.820206 21491 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "636dcc63792e431591309553dd780724" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f82073061a834a5a829f914b1f9071b3"
I20251023 02:10:31.820258 21640 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "636dcc63792e431591309553dd780724" candidate_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db"
I20251023 02:10:31.820391 21491 raft_consensus.cc:3060] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.820452 21640 raft_consensus.cc:3060] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:31.822399 21491 raft_consensus.cc:2468] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 1.
I20251023 02:10:31.822400 21640 raft_consensus.cc:2468] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfdeb364bed4ee5acbd9f5d365e0aef in term 1.
I20251023 02:10:31.822748 21531 leader_election.cc:304] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6f3b5fe80b4fae8ebe4328a8fb67db, ddfdeb364bed4ee5acbd9f5d365e0aef; no voters:
I20251023 02:10:31.822961 21691 raft_consensus.cc:2804] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:31.823087 21691 raft_consensus.cc:697] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Becoming Leader. State: Replica: ddfdeb364bed4ee5acbd9f5d365e0aef, State: Running, Role: LEADER
I20251023 02:10:31.823297 21691 consensus_queue.cc:237] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } }
I20251023 02:10:31.825073 21388 catalog_manager.cc:5649] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef reported cstate change: term changed from 0 to 1, leader changed from <none> to ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194). New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:31.866737 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:10:31.884574 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:31.884843 16007 tablet_replica.cc:333] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3: stopping tablet replica
I20251023 02:10:31.884989 16007 raft_consensus.cc:2243] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:31.885110 16007 raft_consensus.cc:2272] T 636dcc63792e431591309553dd780724 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:31.885614 16007 tablet_replica.cc:333] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3: stopping tablet replica
I20251023 02:10:31.885730 16007 raft_consensus.cc:2243] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:31.885839 16007 raft_consensus.cc:2272] T 5f73f4ecf0114cf19b74ecdff72bad50 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:31.886267 16007 tablet_replica.cc:333] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3: stopping tablet replica
I20251023 02:10:31.886376 16007 raft_consensus.cc:2243] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:31.886477 16007 raft_consensus.cc:2272] T d8b104894f3b48e48218fd262ada68c8 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:31.886902 16007 tablet_replica.cc:333] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3: stopping tablet replica
I20251023 02:10:31.887024 16007 raft_consensus.cc:2243] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:31.887130 16007 raft_consensus.cc:2272] T 92c1cc41f8a34158ab49084fc756f5f1 P f82073061a834a5a829f914b1f9071b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:31.900660 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
W20251023 02:10:32.213238 21531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20251023 02:10:32.214200 21531 consensus_peers.cc:597] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:32.216457 21691 consensus_queue.cc:1048] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:10:32.240060 21531 consensus_peers.cc:597] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:32.252530 21690 consensus_queue.cc:1048] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:10:32.256819 21605 consensus_peers.cc:597] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:32.330636 21691 consensus_queue.cc:1048] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:32.343000 21718 consensus_queue.cc:1048] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:10:32.346447 21605 consensus_peers.cc:597] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:32.892807 21709 consensus_queue.cc:579] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.104s)
W20251023 02:10:32.895339 21389 catalog_manager.cc:5327] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 92c1cc41f8a34158ab49084fc756f5f1 with cas_config_opid_index -1: no extra replica candidate found for tablet 92c1cc41f8a34158ab49084fc756f5f1 (table test-workload [id=a64064023f9a49908ca9c3400601aa71]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20251023 02:10:32.904210 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:32.906287 21727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:32.906685 21728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:32.906934 21730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:32.906957 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:32.907222 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:32.907290 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:32.907325 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185432907321 us; error 0 us; skew 500 ppm
I20251023 02:10:32.907460 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:32.908687 16007 webserver.cc:492] Webserver started at http://127.15.161.196:33025/ using document root <none> and password file <none>
I20251023 02:10:32.908910 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:32.908967 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:32.909078 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:32.909600 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "f7c3200b133c422b8c73a623b2dff200"
format_stamp: "Formatted at 2025-10-23 02:10:32 on dist-test-slave-bfbt"
I20251023 02:10:32.911901 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:32.913174 21735 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:32.913432 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:32.913532 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "f7c3200b133c422b8c73a623b2dff200"
format_stamp: "Formatted at 2025-10-23 02:10:32 on dist-test-slave-bfbt"
I20251023 02:10:32.913611 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:32.945490 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:32.945945 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:32.946412 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:32.947067 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:32.947114 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:32.947177 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:32.947211 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:32.956391 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:40905
I20251023 02:10:32.956411 21797 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:40905 every 8 connection(s)
I20251023 02:10:32.966862 21798 heartbeater.cc:344] Connected to a master server at 127.15.161.254:41493
I20251023 02:10:32.966950 21798 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:32.967168 21798 heartbeater.cc:507] Master 127.15.161.254:41493 requested a full tablet report, sending...
I20251023 02:10:32.967782 21389 ts_manager.cc:194] Registered new tserver with Master: f7c3200b133c422b8c73a623b2dff200 (127.15.161.196:40905)
I20251023 02:10:32.968432 21389 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36338
I20251023 02:10:33.007421 21640 consensus_queue.cc:237] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.009394 21566 raft_consensus.cc:1275] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:33.009960 21725 consensus_queue.cc:1048] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 02:10:33.010142 21605 consensus_peers.cc:597] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.011827 21726 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.012308 21566 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
W20251023 02:10:33.014922 21604 consensus_peers.cc:597] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f7c3200b133c422b8c73a623b2dff200 (127.15.161.196:40905): Couldn't send request to peer f7c3200b133c422b8c73a623b2dff200. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92c1cc41f8a34158ab49084fc756f5f1. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.015069 21375 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 92c1cc41f8a34158ab49084fc756f5f1 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
I20251023 02:10:33.015987 21389 catalog_manager.cc:5649] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef reported cstate change: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.108794 21705 consensus_queue.cc:579] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.285s)
I20251023 02:10:33.111887 21566 consensus_queue.cc:237] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.114090 21640 raft_consensus.cc:1275] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:33.114496 21808 consensus_queue.cc:1048] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 02:10:33.114630 21531 consensus_peers.cc:597] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.116214 21809 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.116695 21640 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
W20251023 02:10:33.118501 21530 consensus_peers.cc:597] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f7c3200b133c422b8c73a623b2dff200 (127.15.161.196:40905): Couldn't send request to peer f7c3200b133c422b8c73a623b2dff200. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 636dcc63792e431591309553dd780724. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.119377 21375 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 636dcc63792e431591309553dd780724 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 02:10:33.120332 21388 catalog_manager.cc:5649] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef reported cstate change: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:33.184000 21808 consensus_queue.cc:579] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.456s)
I20251023 02:10:33.186272 21566 consensus_queue.cc:237] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.187655 21640 raft_consensus.cc:1275] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 02:10:33.187667 21530 consensus_peers.cc:597] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f7c3200b133c422b8c73a623b2dff200 (127.15.161.196:40905): Couldn't send request to peer f7c3200b133c422b8c73a623b2dff200. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8b104894f3b48e48218fd262ada68c8. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.188055 21809 consensus_queue.cc:1048] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 02:10:33.188446 21531 consensus_peers.cc:597] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.189385 21808 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.189868 21640 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.191758 21375 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d8b104894f3b48e48218fd262ada68c8 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 02:10:33.192847 21389 catalog_manager.cc:5649] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef reported cstate change: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:33.276136 21726 consensus_queue.cc:579] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.529s)
I20251023 02:10:33.278880 21640 consensus_queue.cc:237] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.280727 21566 raft_consensus.cc:1275] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 02:10:33.280735 21604 consensus_peers.cc:597] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f7c3200b133c422b8c73a623b2dff200 (127.15.161.196:40905): Couldn't send request to peer f7c3200b133c422b8c73a623b2dff200. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f73f4ecf0114cf19b74ecdff72bad50. This is attempt 1: this message will repeat every 5th retry.
W20251023 02:10:33.281385 21605 consensus_peers.cc:597] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:33.281404 21709 consensus_queue.cc:1048] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:33.283116 21726 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.283569 21566 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } } }
I20251023 02:10:33.286047 21375 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5f73f4ecf0114cf19b74ecdff72bad50 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 02:10:33.287081 21388 catalog_manager.cc:5649] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: config changed from index -1 to 2, NON_VOTER f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 02:10:33.392719 21434 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver f82073061a834a5a829f914b1f9071b3 not available for placement
I20251023 02:10:33.466818 21813 ts_tablet_manager.cc:933] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: Initiating tablet copy from peer 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609)
I20251023 02:10:33.467279 21813 tablet_copy_client.cc:323] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:45609
I20251023 02:10:33.474292 21650 tablet_copy_service.cc:140] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Received BeginTabletCopySession request for tablet 92c1cc41f8a34158ab49084fc756f5f1 from peer f7c3200b133c422b8c73a623b2dff200 ({username='slave'} at 127.0.0.1:39728)
I20251023 02:10:33.474396 21650 tablet_copy_service.cc:161] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Beginning new tablet copy session on tablet 92c1cc41f8a34158ab49084fc756f5f1 from peer f7c3200b133c422b8c73a623b2dff200 at {username='slave'} at 127.0.0.1:39728: session id = f7c3200b133c422b8c73a623b2dff200-92c1cc41f8a34158ab49084fc756f5f1
I20251023 02:10:33.476058 21650 tablet_copy_source_session.cc:215] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:33.476928 21813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c1cc41f8a34158ab49084fc756f5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:33.480312 21813 tablet_copy_client.cc:806] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:33.480538 21813 tablet_copy_client.cc:670] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:33.481839 21813 tablet_copy_client.cc:538] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:33.483875 21813 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: Bootstrap starting.
I20251023 02:10:33.487524 21813 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:33.487792 21813 tablet_bootstrap.cc:492] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: Bootstrap complete.
I20251023 02:10:33.487941 21813 ts_tablet_manager.cc:1403] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 02:10:33.488363 21813 raft_consensus.cc:359] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.488481 21813 raft_consensus.cc:740] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f7c3200b133c422b8c73a623b2dff200, State: Initialized, Role: LEARNER
I20251023 02:10:33.488588 21813 consensus_queue.cc:260] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.489077 21798 heartbeater.cc:499] Master 127.15.161.254:41493 was elected leader, sending a full tablet report...
I20251023 02:10:33.489073 21813 ts_tablet_manager.cc:1434] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:33.489627 21650 tablet_copy_service.cc:342] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Request end of tablet copy session f7c3200b133c422b8c73a623b2dff200-92c1cc41f8a34158ab49084fc756f5f1 received from {username='slave'} at 127.0.0.1:39728
I20251023 02:10:33.489701 21650 tablet_copy_service.cc:434] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: ending tablet copy session f7c3200b133c422b8c73a623b2dff200-92c1cc41f8a34158ab49084fc756f5f1 on tablet 92c1cc41f8a34158ab49084fc756f5f1 with peer f7c3200b133c422b8c73a623b2dff200
I20251023 02:10:33.678624 21813 ts_tablet_manager.cc:933] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: Initiating tablet copy from peer 0b6f3b5fe80b4fae8ebe4328a8fb67db (127.15.161.195:45609)
I20251023 02:10:33.679103 21813 tablet_copy_client.cc:323] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:45609
I20251023 02:10:33.679592 21650 tablet_copy_service.cc:140] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Received BeginTabletCopySession request for tablet 5f73f4ecf0114cf19b74ecdff72bad50 from peer f7c3200b133c422b8c73a623b2dff200 ({username='slave'} at 127.0.0.1:39728)
I20251023 02:10:33.679692 21650 tablet_copy_service.cc:161] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Beginning new tablet copy session on tablet 5f73f4ecf0114cf19b74ecdff72bad50 from peer f7c3200b133c422b8c73a623b2dff200 at {username='slave'} at 127.0.0.1:39728: session id = f7c3200b133c422b8c73a623b2dff200-5f73f4ecf0114cf19b74ecdff72bad50
I20251023 02:10:33.681517 21650 tablet_copy_source_session.cc:215] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:33.682412 21813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f73f4ecf0114cf19b74ecdff72bad50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:33.685973 21813 tablet_copy_client.cc:806] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:33.686182 21813 tablet_copy_client.cc:670] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:33.687256 21813 tablet_copy_client.cc:538] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:33.687526 21820 ts_tablet_manager.cc:933] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: Initiating tablet copy from peer ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:33.687942 21820 tablet_copy_client.cc:323] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:38737
I20251023 02:10:33.690079 21813 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: Bootstrap starting.
I20251023 02:10:33.693518 21576 tablet_copy_service.cc:140] P ddfdeb364bed4ee5acbd9f5d365e0aef: Received BeginTabletCopySession request for tablet d8b104894f3b48e48218fd262ada68c8 from peer f7c3200b133c422b8c73a623b2dff200 ({username='slave'} at 127.0.0.1:39890)
I20251023 02:10:33.693642 21576 tablet_copy_service.cc:161] P ddfdeb364bed4ee5acbd9f5d365e0aef: Beginning new tablet copy session on tablet d8b104894f3b48e48218fd262ada68c8 from peer f7c3200b133c422b8c73a623b2dff200 at {username='slave'} at 127.0.0.1:39890: session id = f7c3200b133c422b8c73a623b2dff200-d8b104894f3b48e48218fd262ada68c8
I20251023 02:10:33.695052 21813 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:33.695428 21813 tablet_bootstrap.cc:492] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: Bootstrap complete.
I20251023 02:10:33.695572 21576 tablet_copy_source_session.cc:215] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:33.695648 21813 ts_tablet_manager.cc:1403] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20251023 02:10:33.696241 21813 raft_consensus.cc:359] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.696451 21820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8b104894f3b48e48218fd262ada68c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:33.696426 21813 raft_consensus.cc:740] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f7c3200b133c422b8c73a623b2dff200, State: Initialized, Role: LEARNER
I20251023 02:10:33.696619 21813 consensus_queue.cc:260] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.697104 21813 ts_tablet_manager.cc:1434] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251023 02:10:33.697585 21650 tablet_copy_service.cc:342] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: Request end of tablet copy session f7c3200b133c422b8c73a623b2dff200-5f73f4ecf0114cf19b74ecdff72bad50 received from {username='slave'} at 127.0.0.1:39728
I20251023 02:10:33.697669 21650 tablet_copy_service.cc:434] P 0b6f3b5fe80b4fae8ebe4328a8fb67db: ending tablet copy session f7c3200b133c422b8c73a623b2dff200-5f73f4ecf0114cf19b74ecdff72bad50 on tablet 5f73f4ecf0114cf19b74ecdff72bad50 with peer f7c3200b133c422b8c73a623b2dff200
I20251023 02:10:33.697918 21824 ts_tablet_manager.cc:933] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: Initiating tablet copy from peer ddfdeb364bed4ee5acbd9f5d365e0aef (127.15.161.194:38737)
I20251023 02:10:33.698503 21824 tablet_copy_client.cc:323] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:38737
I20251023 02:10:33.699079 21576 tablet_copy_service.cc:140] P ddfdeb364bed4ee5acbd9f5d365e0aef: Received BeginTabletCopySession request for tablet 636dcc63792e431591309553dd780724 from peer f7c3200b133c422b8c73a623b2dff200 ({username='slave'} at 127.0.0.1:39890)
I20251023 02:10:33.699185 21576 tablet_copy_service.cc:161] P ddfdeb364bed4ee5acbd9f5d365e0aef: Beginning new tablet copy session on tablet 636dcc63792e431591309553dd780724 from peer f7c3200b133c422b8c73a623b2dff200 at {username='slave'} at 127.0.0.1:39890: session id = f7c3200b133c422b8c73a623b2dff200-636dcc63792e431591309553dd780724
I20251023 02:10:33.701249 21576 tablet_copy_source_session.cc:215] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:33.701304 21820 tablet_copy_client.cc:806] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:33.701452 21820 tablet_copy_client.cc:670] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:33.702210 21824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636dcc63792e431591309553dd780724. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:33.702839 21820 tablet_copy_client.cc:538] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:33.704816 21820 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: Bootstrap starting.
I20251023 02:10:33.705672 21824 tablet_copy_client.cc:806] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:33.705847 21824 tablet_copy_client.cc:670] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:33.707037 21824 tablet_copy_client.cc:538] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:33.708835 21820 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:33.709100 21820 tablet_bootstrap.cc:492] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: Bootstrap complete.
I20251023 02:10:33.709270 21820 ts_tablet_manager.cc:1403] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 02:10:33.709343 21824 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: Bootstrap starting.
I20251023 02:10:33.709748 21820 raft_consensus.cc:359] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.709863 21820 raft_consensus.cc:740] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f7c3200b133c422b8c73a623b2dff200, State: Initialized, Role: LEARNER
I20251023 02:10:33.709990 21820 consensus_queue.cc:260] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.710319 21820 ts_tablet_manager.cc:1434] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: Time spent starting tablet: real 0.001s user 0.000s sys 0.003s
I20251023 02:10:33.711035 21576 tablet_copy_service.cc:342] P ddfdeb364bed4ee5acbd9f5d365e0aef: Request end of tablet copy session f7c3200b133c422b8c73a623b2dff200-d8b104894f3b48e48218fd262ada68c8 received from {username='slave'} at 127.0.0.1:39890
I20251023 02:10:33.711136 21576 tablet_copy_service.cc:434] P ddfdeb364bed4ee5acbd9f5d365e0aef: ending tablet copy session f7c3200b133c422b8c73a623b2dff200-d8b104894f3b48e48218fd262ada68c8 on tablet d8b104894f3b48e48218fd262ada68c8 with peer f7c3200b133c422b8c73a623b2dff200
I20251023 02:10:33.713851 21824 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:33.714069 21824 tablet_bootstrap.cc:492] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: Bootstrap complete.
I20251023 02:10:33.714246 21824 ts_tablet_manager.cc:1403] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: Time spent bootstrapping tablet: real 0.005s user 0.006s sys 0.001s
I20251023 02:10:33.714677 21824 raft_consensus.cc:359] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.714819 21824 raft_consensus.cc:740] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f7c3200b133c422b8c73a623b2dff200, State: Initialized, Role: LEARNER
I20251023 02:10:33.714943 21824 consensus_queue.cc:260] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: true } }
I20251023 02:10:33.715242 21824 ts_tablet_manager.cc:1434] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:33.715685 21576 tablet_copy_service.cc:342] P ddfdeb364bed4ee5acbd9f5d365e0aef: Request end of tablet copy session f7c3200b133c422b8c73a623b2dff200-636dcc63792e431591309553dd780724 received from {username='slave'} at 127.0.0.1:39890
I20251023 02:10:33.715785 21576 tablet_copy_service.cc:434] P ddfdeb364bed4ee5acbd9f5d365e0aef: ending tablet copy session f7c3200b133c422b8c73a623b2dff200-636dcc63792e431591309553dd780724 on tablet 636dcc63792e431591309553dd780724 with peer f7c3200b133c422b8c73a623b2dff200
I20251023 02:10:34.076711 21773 raft_consensus.cc:1217] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 02:10:34.145586 21773 raft_consensus.cc:1217] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 02:10:34.238351 21808 consensus_queue.cc:579] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.052s)
I20251023 02:10:34.269860 21773 raft_consensus.cc:1217] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 02:10:34.307210 21773 raft_consensus.cc:1217] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20251023 02:10:34.393361 21434 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver f82073061a834a5a829f914b1f9071b3 not available for placement
I20251023 02:10:34.418519 21831 consensus_queue.cc:579] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.306s)
I20251023 02:10:34.505649 21828 consensus_queue.cc:579] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.498s)
I20251023 02:10:34.569311 21829 consensus_queue.cc:579] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Leader has been unable to successfully communicate with peer f82073061a834a5a829f914b1f9071b3 for more than 1 seconds (1.290s)
I20251023 02:10:34.577751 21829 raft_consensus.cc:1064] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: attempting to promote NON_VOTER f7c3200b133c422b8c73a623b2dff200 to VOTER
I20251023 02:10:34.578332 21829 consensus_queue.cc:237] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.579967 21566 raft_consensus.cc:1275] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:34.579967 21773 raft_consensus.cc:1275] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:34.580441 21833 consensus_queue.cc:1048] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.580709 21829 consensus_queue.cc:1048] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251023 02:10:34.580943 21605 consensus_peers.cc:597] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:34.583274 21833 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.583791 21566 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.586082 21773 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.586843 21389 catalog_manager.cc:5649] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:10:34.591593 21640 consensus_queue.cc:237] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.592934 21773 raft_consensus.cc:1275] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.593314 21829 consensus_queue.cc:1048] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.593858 21566 raft_consensus.cc:1275] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.594236 21829 consensus_queue.cc:1048] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.595088 21833 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.595619 21773 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.597968 21375 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 92c1cc41f8a34158ab49084fc756f5f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:34.598484 21566 raft_consensus.cc:2955] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.598987 21387 catalog_manager.cc:5649] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20251023 02:10:34.603226 21375 catalog_manager.cc:4712] TS f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 92c1cc41f8a34158ab49084fc756f5f1: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111)
I20251023 02:10:34.660368 21829 raft_consensus.cc:1064] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: attempting to promote NON_VOTER f7c3200b133c422b8c73a623b2dff200 to VOTER
I20251023 02:10:34.661022 21829 consensus_queue.cc:237] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.662750 21566 raft_consensus.cc:1275] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:34.663013 21773 raft_consensus.cc:1275] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20251023 02:10:34.663139 21605 consensus_peers.cc:597] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:34.663218 21828 consensus_queue.cc:1048] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.663411 21829 consensus_queue.cc:1048] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.665535 21829 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.666013 21773 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.668134 21566 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.668669 21387 catalog_manager.cc:5649] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:10:34.672242 21640 consensus_queue.cc:237] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.673575 21566 raft_consensus.cc:1275] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.673861 21773 raft_consensus.cc:1275] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Refusing update from remote peer 0b6f3b5fe80b4fae8ebe4328a8fb67db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.673995 21828 consensus_queue.cc:1048] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.674424 21828 consensus_queue.cc:1048] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.675583 21726 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.676453 21773 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.677829 21375 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 5f73f4ecf0114cf19b74ecdff72bad50 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:34.677991 21566 raft_consensus.cc:2955] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.678741 21389 catalog_manager.cc:5649] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:10:34.708252 21830 raft_consensus.cc:1064] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: attempting to promote NON_VOTER f7c3200b133c422b8c73a623b2dff200 to VOTER
I20251023 02:10:34.708668 21831 raft_consensus.cc:1064] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: attempting to promote NON_VOTER f7c3200b133c422b8c73a623b2dff200 to VOTER
I20251023 02:10:34.708811 21830 consensus_queue.cc:237] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.709220 21831 consensus_queue.cc:237] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.710641 21640 raft_consensus.cc:1275] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:34.710781 21773 raft_consensus.cc:1275] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:34.711544 21808 consensus_queue.cc:1048] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.711768 21831 consensus_queue.cc:1048] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.712330 21773 raft_consensus.cc:1275] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 LEARNER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20251023 02:10:34.712626 21531 consensus_peers.cc:597] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:34.712653 21640 raft_consensus.cc:1275] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20251023 02:10:34.712724 21531 consensus_peers.cc:597] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef -> Peer f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): Couldn't send request to peer f82073061a834a5a829f914b1f9071b3. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:34.713042 21831 consensus_queue.cc:1048] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.713699 21830 consensus_queue.cc:1048] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:34.715695 21831 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.715788 21808 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.718298 21773 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.718410 21772 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.718554 21640 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.719308 21639 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.721589 21387 catalog_manager.cc:5649] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef reported cstate change: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:10:34.724464 21388 catalog_manager.cc:5649] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db reported cstate change: config changed from index 2 to 3, f7c3200b133c422b8c73a623b2dff200 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f82073061a834a5a829f914b1f9071b3" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 34467 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.725232 21566 consensus_queue.cc:237] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.726612 21773 raft_consensus.cc:1275] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.726778 21639 raft_consensus.cc:1275] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.727175 21846 consensus_queue.cc:1048] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.727348 21853 consensus_queue.cc:1048] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.729359 21853 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.729921 21773 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.730814 21640 raft_consensus.cc:2955] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.731285 21566 consensus_queue.cc:237] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } }
I20251023 02:10:34.732790 21375 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 636dcc63792e431591309553dd780724 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:34.732816 21638 raft_consensus.cc:1275] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.733335 21772 raft_consensus.cc:1275] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Refusing update from remote peer ddfdeb364bed4ee5acbd9f5d365e0aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:34.733565 21853 consensus_queue.cc:1048] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.733880 21830 consensus_queue.cc:1048] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:34.734899 21389 catalog_manager.cc:5649] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 reported cstate change: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.735752 21830 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.736563 21772 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.738536 21638 raft_consensus.cc:2955] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.740190 21388 catalog_manager.cc:5649] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 reported cstate change: config changed from index 3 to 4, VOTER f82073061a834a5a829f914b1f9071b3 (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ddfdeb364bed4ee5acbd9f5d365e0aef" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 38737 } } peers { permanent_uuid: "0b6f3b5fe80b4fae8ebe4328a8fb67db" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45609 } } peers { permanent_uuid: "f7c3200b133c422b8c73a623b2dff200" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 40905 } attrs { promote: false } } }
I20251023 02:10:34.742203 21375 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet d8b104894f3b48e48218fd262ada68c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20251023 02:10:35.799166 21375 catalog_manager.cc:4712] TS f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 92c1cc41f8a34158ab49084fc756f5f1: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111)
W20251023 02:10:36.851778 21375 catalog_manager.cc:4712] TS f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 92c1cc41f8a34158ab49084fc756f5f1: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111)
W20251023 02:10:38.915400 21375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111) [suppressed 61 similar messages]
W20251023 02:10:38.916324 21375 catalog_manager.cc:4712] TS f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 92c1cc41f8a34158ab49084fc756f5f1: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111)
I20251023 02:10:39.983928 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:10:39.994374 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:39.994567 16007 tablet_replica.cc:333] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef: stopping tablet replica
I20251023 02:10:39.994724 16007 raft_consensus.cc:2243] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:39.994930 16007 raft_consensus.cc:2272] T 92c1cc41f8a34158ab49084fc756f5f1 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:39.995462 16007 tablet_replica.cc:333] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef: stopping tablet replica
I20251023 02:10:39.995584 16007 raft_consensus.cc:2243] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:39.995815 16007 raft_consensus.cc:2272] T d8b104894f3b48e48218fd262ada68c8 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:39.996189 16007 tablet_replica.cc:333] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef: stopping tablet replica
I20251023 02:10:39.996297 16007 raft_consensus.cc:2243] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:39.996462 16007 raft_consensus.cc:2272] T 5f73f4ecf0114cf19b74ecdff72bad50 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:39.996810 16007 tablet_replica.cc:333] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef: stopping tablet replica
I20251023 02:10:39.996913 16007 raft_consensus.cc:2243] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:39.997092 16007 raft_consensus.cc:2272] T 636dcc63792e431591309553dd780724 P ddfdeb364bed4ee5acbd9f5d365e0aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.011695 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:10:40.014864 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:10:40.022188 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:40.022426 16007 tablet_replica.cc:333] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: stopping tablet replica
I20251023 02:10:40.022563 16007 raft_consensus.cc:2243] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:40.022775 16007 raft_consensus.cc:2272] T 92c1cc41f8a34158ab49084fc756f5f1 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.023160 16007 tablet_replica.cc:333] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: stopping tablet replica
I20251023 02:10:40.023291 16007 raft_consensus.cc:2243] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:40.023470 16007 raft_consensus.cc:2272] T d8b104894f3b48e48218fd262ada68c8 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.023826 16007 tablet_replica.cc:333] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: stopping tablet replica
I20251023 02:10:40.023948 16007 raft_consensus.cc:2243] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:40.024127 16007 raft_consensus.cc:2272] T 636dcc63792e431591309553dd780724 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.024473 16007 tablet_replica.cc:333] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db: stopping tablet replica
I20251023 02:10:40.024590 16007 raft_consensus.cc:2243] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:40.024793 16007 raft_consensus.cc:2272] T 5f73f4ecf0114cf19b74ecdff72bad50 P 0b6f3b5fe80b4fae8ebe4328a8fb67db [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.037850 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:10:40.040601 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:10:40.048388 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:40.048584 16007 tablet_replica.cc:333] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200: stopping tablet replica
I20251023 02:10:40.048740 16007 raft_consensus.cc:2243] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:40.048887 16007 raft_consensus.cc:2272] T d8b104894f3b48e48218fd262ada68c8 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.049226 16007 tablet_replica.cc:333] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200: stopping tablet replica
I20251023 02:10:40.049326 16007 raft_consensus.cc:2243] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:40.049458 16007 raft_consensus.cc:2272] T 92c1cc41f8a34158ab49084fc756f5f1 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.049794 16007 tablet_replica.cc:333] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200: stopping tablet replica
I20251023 02:10:40.049890 16007 raft_consensus.cc:2243] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:40.050024 16007 raft_consensus.cc:2272] T 636dcc63792e431591309553dd780724 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.050390 16007 tablet_replica.cc:333] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200: stopping tablet replica
I20251023 02:10:40.050482 16007 raft_consensus.cc:2243] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:40.050614 16007 raft_consensus.cc:2272] T 5f73f4ecf0114cf19b74ecdff72bad50 P f7c3200b133c422b8c73a623b2dff200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:40.063735 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:10:40.077229 16007 master.cc:561] Master@127.15.161.254:41493 shutting down...
W20251023 02:10:43.025792 21375 catalog_manager.cc:4712] TS f82073061a834a5a829f914b1f9071b3 (127.15.161.193:34467): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 92c1cc41f8a34158ab49084fc756f5f1: Network error: Client connection negotiation failed: client connection to 127.15.161.193:34467: connect: Connection refused (error 111)
I20251023 02:10:43.675782 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:43.676031 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:43.676127 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 939799739d5e4f6fb69056e4a5629024: stopping tablet replica
I20251023 02:10:43.694278 16007 master.cc:583] Master@127.15.161.254:41493 shutdown complete.
[ OK ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering (13368 ms)
[ RUN ] AutoRebalancerTest.TestHandlingFailedTservers
I20251023 02:10:43.707119 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:34009
I20251023 02:10:43.707569 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:43.709649 21861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.709692 21860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.709803 21863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:43.710104 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:43.710299 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:43.710369 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:43.710403 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185443710398 us; error 0 us; skew 500 ppm
I20251023 02:10:43.710512 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:43.711464 16007 webserver.cc:492] Webserver started at http://127.15.161.254:38303/ using document root <none> and password file <none>
I20251023 02:10:43.711691 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:43.711741 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:43.711824 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:43.712361 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "fd187014ebc04229bef4615b248db85a"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.714573 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:43.715883 21868 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.716159 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:43.716229 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "fd187014ebc04229bef4615b248db85a"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.716324 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:43.731848 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:43.732213 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:43.740619 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:34009
I20251023 02:10:43.740646 21919 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:34009 every 8 connection(s)
I20251023 02:10:43.741655 21920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:43.745246 21920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a: Bootstrap starting.
I20251023 02:10:43.746203 21920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:43.747380 21920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a: No bootstrap required, opened a new log
I20251023 02:10:43.747835 21920 raft_consensus.cc:359] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd187014ebc04229bef4615b248db85a" member_type: VOTER }
I20251023 02:10:43.747936 21920 raft_consensus.cc:385] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:43.747985 21920 raft_consensus.cc:740] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd187014ebc04229bef4615b248db85a, State: Initialized, Role: FOLLOWER
I20251023 02:10:43.748147 21920 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd187014ebc04229bef4615b248db85a" member_type: VOTER }
I20251023 02:10:43.748239 21920 raft_consensus.cc:399] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:10:43.748307 21920 raft_consensus.cc:493] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:10:43.748381 21920 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:43.749862 21920 raft_consensus.cc:515] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd187014ebc04229bef4615b248db85a" member_type: VOTER }
I20251023 02:10:43.750015 21920 leader_election.cc:304] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd187014ebc04229bef4615b248db85a; no voters:
I20251023 02:10:43.750329 21920 leader_election.cc:290] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:10:43.750425 21923 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:43.750865 21923 raft_consensus.cc:697] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 1 LEADER]: Becoming Leader. State: Replica: fd187014ebc04229bef4615b248db85a, State: Running, Role: LEADER
I20251023 02:10:43.751042 21923 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd187014ebc04229bef4615b248db85a" member_type: VOTER }
I20251023 02:10:43.751081 21920 sys_catalog.cc:565] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:10:43.751694 21925 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd187014ebc04229bef4615b248db85a. Latest consensus state: current_term: 1 leader_uuid: "fd187014ebc04229bef4615b248db85a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd187014ebc04229bef4615b248db85a" member_type: VOTER } }
I20251023 02:10:43.751659 21924 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd187014ebc04229bef4615b248db85a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd187014ebc04229bef4615b248db85a" member_type: VOTER } }
I20251023 02:10:43.751854 21925 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:43.751952 21924 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:43.754427 21928 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:10:43.755579 21928 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:10:43.755935 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:10:43.757854 21928 catalog_manager.cc:1357] Generated new cluster ID: d1f97d3d73684b919a66ac1735fd911e
I20251023 02:10:43.757915 21928 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:10:43.784050 21928 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:10:43.784649 21928 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:10:43.789597 21928 catalog_manager.cc:6022] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a: Generated new TSK 0
I20251023 02:10:43.789757 21928 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:10:43.820392 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:43.822510 21941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.822868 21942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:43.823437 16007 server_base.cc:1047] running on GCE node
W20251023 02:10:43.823625 21944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:43.823856 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:43.823912 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:43.823956 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185443823940 us; error 0 us; skew 500 ppm
I20251023 02:10:43.824060 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:43.825008 16007 webserver.cc:492] Webserver started at http://127.15.161.193:43935/ using document root <none> and password file <none>
I20251023 02:10:43.825186 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:43.825242 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:43.825351 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:43.825868 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "9165a251458341aebec6b4ece6f128e1"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.828018 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:43.829339 21949 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.829609 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:43.829694 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "9165a251458341aebec6b4ece6f128e1"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.829775 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:43.836815 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:43.837213 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:43.837646 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:43.838418 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:43.838470 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.838575 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:43.838619 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.849771 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:39333
I20251023 02:10:43.849808 22011 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:39333 every 8 connection(s)
I20251023 02:10:43.851223 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:43.854033 22016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.854492 22017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.854652 22019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:43.855087 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:43.855326 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:43.855382 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:43.855432 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185443855427 us; error 0 us; skew 500 ppm
I20251023 02:10:43.855593 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:43.856810 16007 webserver.cc:492] Webserver started at http://127.15.161.194:33995/ using document root <none> and password file <none>
I20251023 02:10:43.857014 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:43.857076 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:43.857192 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:43.857753 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "87e7fa17045341a4ba9533cada498b31"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.860134 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:43.861500 22024 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.861878 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:43.861984 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "87e7fa17045341a4ba9533cada498b31"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.862092 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:43.863076 22012 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:43.863184 22012 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:43.863562 22012 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:43.864459 21885 ts_manager.cc:194] Registered new tserver with Master: 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333)
I20251023 02:10:43.865783 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60376
I20251023 02:10:43.874339 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:43.874673 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:43.875110 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:43.875809 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:43.875859 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.875921 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:43.875957 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.886224 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:39041
I20251023 02:10:43.886273 22086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:39041 every 8 connection(s)
I20251023 02:10:43.888000 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:43.890834 22090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.890939 22091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:43.892072 22093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:43.894771 22087 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:43.894915 22087 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:43.895157 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:43.895349 22087 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:43.895447 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:43.895539 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:43.895594 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185443895574 us; error 0 us; skew 500 ppm
I20251023 02:10:43.895773 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:43.896340 21885 ts_manager.cc:194] Registered new tserver with Master: 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:43.896770 16007 webserver.cc:492] Webserver started at http://127.15.161.195:39229/ using document root <none> and password file <none>
I20251023 02:10:43.896955 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:43.897035 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:43.897156 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:43.897214 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60382
I20251023 02:10:43.897833 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "00b51eaa2b6142f7a0e665109a362654"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.900095 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 02:10:43.901505 22098 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.901881 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:43.901988 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "00b51eaa2b6142f7a0e665109a362654"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:43.902097 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:43.918705 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:43.919101 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:43.919574 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:43.920214 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:43.920280 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.920367 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:43.920415 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:43.930660 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:45859
I20251023 02:10:43.930692 22160 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:45859 every 8 connection(s)
I20251023 02:10:43.936786 22161 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:43.936902 22161 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:43.937263 22161 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:43.937999 21885 ts_manager.cc:194] Registered new tserver with Master: 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:43.938236 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006700505s
I20251023 02:10:43.938913 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60398
I20251023 02:10:44.867026 22012 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
I20251023 02:10:44.898312 22087 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
I20251023 02:10:44.940095 22161 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
I20251023 02:10:44.965582 16007 test_util.cc:276] Using random seed: -254497593
I20251023 02:10:44.979534 21885 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60402:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:10:44.980134 21885 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:10:44.997259 22126 tablet_service.cc:1505] Processing CreateTablet for tablet 76bcdb4dce034db296f8ef2eafb633e5 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:44.997370 22125 tablet_service.cc:1505] Processing CreateTablet for tablet d720c77c125e449dab6cfbe71b57c196 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:44.997681 22123 tablet_service.cc:1505] Processing CreateTablet for tablet 1506a9a3315f483ca6869328f4cc0d0e (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:44.997812 22126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76bcdb4dce034db296f8ef2eafb633e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:44.998248 22124 tablet_service.cc:1505] Processing CreateTablet for tablet 3242b61802d64be189ad59b48a4aa5b8 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:44.998720 22123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506a9a3315f483ca6869328f4cc0d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:44.999226 22124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3242b61802d64be189ad59b48a4aa5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.001930 22052 tablet_service.cc:1505] Processing CreateTablet for tablet 76bcdb4dce034db296f8ef2eafb633e5 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:45.002017 22051 tablet_service.cc:1505] Processing CreateTablet for tablet d720c77c125e449dab6cfbe71b57c196 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:45.002368 22052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76bcdb4dce034db296f8ef2eafb633e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.002594 22051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d720c77c125e449dab6cfbe71b57c196. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.002526 22049 tablet_service.cc:1505] Processing CreateTablet for tablet 1506a9a3315f483ca6869328f4cc0d0e (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:45.003033 22050 tablet_service.cc:1505] Processing CreateTablet for tablet 3242b61802d64be189ad59b48a4aa5b8 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:45.003489 22050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3242b61802d64be189ad59b48a4aa5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.006743 22049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506a9a3315f483ca6869328f4cc0d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.007386 21977 tablet_service.cc:1505] Processing CreateTablet for tablet 76bcdb4dce034db296f8ef2eafb633e5 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:45.007771 21977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76bcdb4dce034db296f8ef2eafb633e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.010423 22125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d720c77c125e449dab6cfbe71b57c196. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.010650 21976 tablet_service.cc:1505] Processing CreateTablet for tablet d720c77c125e449dab6cfbe71b57c196 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:45.011196 21976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d720c77c125e449dab6cfbe71b57c196. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.014386 22182 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:45.014537 21975 tablet_service.cc:1505] Processing CreateTablet for tablet 3242b61802d64be189ad59b48a4aa5b8 (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:45.014941 21975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3242b61802d64be189ad59b48a4aa5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.016109 22182 tablet_bootstrap.cc:654] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.017660 22181 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:45.018536 21974 tablet_service.cc:1505] Processing CreateTablet for tablet 1506a9a3315f483ca6869328f4cc0d0e (DEFAULT_TABLE table=test-workload [id=81a4b8ac9362405d8524fd8b19bdddf6]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:45.019219 22181 tablet_bootstrap.cc:654] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.019729 21974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506a9a3315f483ca6869328f4cc0d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:45.028998 22185 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:45.029913 22182 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: No bootstrap required, opened a new log
I20251023 02:10:45.030104 22182 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20251023 02:10:45.031076 22185 tablet_bootstrap.cc:654] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.030961 22182 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.031307 22182 raft_consensus.cc:385] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.031431 22182 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.031734 22182 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.034615 22182 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20251023 02:10:45.034893 22182 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:45.034950 22181 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: No bootstrap required, opened a new log
I20251023 02:10:45.035130 22181 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.003s
I20251023 02:10:45.035921 22181 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.036125 22181 raft_consensus.cc:385] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.036202 22182 tablet_bootstrap.cc:654] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.036237 22181 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.036945 22181 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.037822 22182 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: No bootstrap required, opened a new log
I20251023 02:10:45.037966 22182 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20251023 02:10:45.038049 22185 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: No bootstrap required, opened a new log
I20251023 02:10:45.038264 22185 ts_tablet_manager.cc:1403] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20251023 02:10:45.038724 22182 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.038888 22182 raft_consensus.cc:385] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.038967 22182 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.039127 22185 raft_consensus.cc:359] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.039194 22182 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.039283 22185 raft_consensus.cc:385] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.039445 22185 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.039770 22182 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:10:45.039692 22185 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.040087 22182 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:45.042117 22182 tablet_bootstrap.cc:654] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.044485 22181 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20251023 02:10:45.044919 22185 ts_tablet_manager.cc:1434] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20251023 02:10:45.045274 22185 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:45.046594 22181 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:45.047156 22185 tablet_bootstrap.cc:654] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.049719 22181 tablet_bootstrap.cc:654] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.051378 22182 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: No bootstrap required, opened a new log
I20251023 02:10:45.051535 22182 ts_tablet_manager.cc:1403] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20251023 02:10:45.051815 22185 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: No bootstrap required, opened a new log
I20251023 02:10:45.051949 22185 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251023 02:10:45.052354 22182 raft_consensus.cc:359] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.052567 22182 raft_consensus.cc:385] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.052665 22182 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.052711 22185 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.052919 22185 raft_consensus.cc:385] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.053025 22185 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.053012 22182 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.053296 22185 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.053603 22182 ts_tablet_manager.cc:1434] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:45.053889 22182 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:45.054404 22185 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:45.054733 22185 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:45.055317 22182 tablet_bootstrap.cc:654] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.056185 22185 tablet_bootstrap.cc:654] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.060887 22181 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: No bootstrap required, opened a new log
I20251023 02:10:45.060923 22182 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: No bootstrap required, opened a new log
I20251023 02:10:45.061035 22181 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20251023 02:10:45.061057 22182 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251023 02:10:45.061307 22185 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: No bootstrap required, opened a new log
I20251023 02:10:45.061436 22185 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251023 02:10:45.061764 22182 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.061919 22182 raft_consensus.cc:385] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.061875 22181 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.062012 22182 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.062052 22181 raft_consensus.cc:385] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.062158 22181 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.062211 22185 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.062367 22185 raft_consensus.cc:385] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.062285 22182 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.062469 22185 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.062449 22181 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.062844 22182 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:45.062860 22185 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.063647 22181 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251023 02:10:45.063962 22181 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:45.064388 22185 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:10:45.064744 22185 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:45.066385 22181 tablet_bootstrap.cc:654] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.066953 22185 tablet_bootstrap.cc:654] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.068331 22181 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: No bootstrap required, opened a new log
I20251023 02:10:45.068486 22181 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251023 02:10:45.068986 22181 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.069126 22181 raft_consensus.cc:385] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.069201 22181 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.069419 22181 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.070914 22185 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: No bootstrap required, opened a new log
I20251023 02:10:45.071074 22185 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251023 02:10:45.071635 22185 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.071823 22185 raft_consensus.cc:385] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.071928 22185 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.072196 22185 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.072562 22185 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:45.087458 22189 raft_consensus.cc:493] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:45.087636 22189 raft_consensus.cc:515] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.088038 22181 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.019s user 0.014s sys 0.006s
I20251023 02:10:45.088277 22181 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:45.088809 22189 leader_election.cc:290] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:45.089705 22181 tablet_bootstrap.cc:654] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:45.093677 22181 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: No bootstrap required, opened a new log
I20251023 02:10:45.093797 22181 ts_tablet_manager.cc:1403] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251023 02:10:45.094488 22181 raft_consensus.cc:359] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.094702 22181 raft_consensus.cc:385] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:45.094803 22181 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:45.095070 22181 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.095489 22181 ts_tablet_manager.cc:1434] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:45.096258 21987 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:45.096426 21987 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 0.
I20251023 02:10:45.096779 22102 leader_election.cc:304] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:45.096995 22198 raft_consensus.cc:2804] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:45.097081 22198 raft_consensus.cc:493] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:45.097172 22198 raft_consensus.cc:3060] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.098444 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:45.098604 22062 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 0.
I20251023 02:10:45.099710 22198 raft_consensus.cc:515] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.100129 22198 leader_election.cc:290] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:45.100387 21987 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:45.100459 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:45.100508 21987 raft_consensus.cc:3060] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.100611 22062 raft_consensus.cc:3060] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Advancing to term 1
W20251023 02:10:45.101920 22013 tablet.cc:2378] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 02:10:45.102442 22062 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:45.102588 21987 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:45.102771 22101 leader_election.cc:304] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 87e7fa17045341a4ba9533cada498b31; no voters:
I20251023 02:10:45.102995 22198 raft_consensus.cc:2804] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:45.103109 22198 raft_consensus.cc:697] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: Becoming Leader. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Running, Role: LEADER
I20251023 02:10:45.103332 22198 consensus_queue.cc:237] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.105394 21885 catalog_manager.cc:5649] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:45.166456 22190 raft_consensus.cc:493] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:45.166653 22190 raft_consensus.cc:515] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.167402 22190 leader_election.cc:290] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041), 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:45.173897 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:45.173904 22136 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654" is_pre_election: true
I20251023 02:10:45.174079 22062 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9165a251458341aebec6b4ece6f128e1 in term 0.
I20251023 02:10:45.174115 22136 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9165a251458341aebec6b4ece6f128e1 in term 0.
I20251023 02:10:45.174467 21952 leader_election.cc:304] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87e7fa17045341a4ba9533cada498b31, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:45.174705 22190 raft_consensus.cc:2804] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:45.174780 22190 raft_consensus.cc:493] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:45.174829 22190 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.176602 22190 raft_consensus.cc:515] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.177109 22190 leader_election.cc:290] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 1 election: Requested vote from peers 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041), 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:45.177510 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:45.177511 22136 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654"
I20251023 02:10:45.177682 22062 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.177743 22136 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.179507 22062 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9165a251458341aebec6b4ece6f128e1 in term 1.
I20251023 02:10:45.179615 22136 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9165a251458341aebec6b4ece6f128e1 in term 1.
I20251023 02:10:45.179872 21952 leader_election.cc:304] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87e7fa17045341a4ba9533cada498b31, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:45.180069 22190 raft_consensus.cc:2804] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:45.180399 22190 raft_consensus.cc:697] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 LEADER]: Becoming Leader. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Running, Role: LEADER
I20251023 02:10:45.180601 22190 consensus_queue.cc:237] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.182904 21885 catalog_manager.cc:5649] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9165a251458341aebec6b4ece6f128e1 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "9165a251458341aebec6b4ece6f128e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:45.183276 22198 raft_consensus.cc:493] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:45.183409 22198 raft_consensus.cc:515] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.183923 22198 leader_election.cc:290] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:45.184165 21987 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:45.184226 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:45.184306 21987 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 0.
I20251023 02:10:45.184360 22062 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 0.
I20251023 02:10:45.184603 22102 leader_election.cc:304] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:45.184818 22198 raft_consensus.cc:2804] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:45.184883 22198 raft_consensus.cc:493] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:45.184949 22198 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.186812 22198 raft_consensus.cc:515] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.187232 22198 leader_election.cc:290] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:45.187531 21987 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:45.187620 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:45.187661 21987 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.187752 22062 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.189343 21987 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:45.189361 22062 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:45.189662 22101 leader_election.cc:304] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 87e7fa17045341a4ba9533cada498b31; no voters:
I20251023 02:10:45.189885 22198 raft_consensus.cc:2804] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:45.189983 22198 raft_consensus.cc:697] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: Becoming Leader. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Running, Role: LEADER
I20251023 02:10:45.190212 22198 consensus_queue.cc:237] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:45.192200 21885 catalog_manager.cc:5649] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:45.273149 22198 raft_consensus.cc:493] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:45.273331 22198 raft_consensus.cc:515] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.273957 22198 leader_election.cc:290] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:45.274437 21987 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:45.274434 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:45.274616 21987 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 0.
I20251023 02:10:45.274636 22062 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 0.
I20251023 02:10:45.274991 22102 leader_election.cc:304] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:45.275219 22198 raft_consensus.cc:2804] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:45.275305 22198 raft_consensus.cc:493] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:45.275369 22198 raft_consensus.cc:3060] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.277031 22198 raft_consensus.cc:515] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.277439 22198 leader_election.cc:290] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:45.277750 21987 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:45.277750 22062 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:45.277915 22062 raft_consensus.cc:3060] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.277915 21987 raft_consensus.cc:3060] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:45.280356 21987 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:45.280356 22062 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:45.280726 22102 leader_election.cc:304] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 87e7fa17045341a4ba9533cada498b31, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:45.280943 22198 raft_consensus.cc:2804] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:45.281051 22198 raft_consensus.cc:697] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: Becoming Leader. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Running, Role: LEADER
I20251023 02:10:45.281243 22198 consensus_queue.cc:237] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:45.283277 21885 catalog_manager.cc:5649] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195). New cstate: current_term: 1 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:45.300760 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:10:45.307807 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:45.308091 16007 tablet_replica.cc:333] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: stopping tablet replica
I20251023 02:10:45.308279 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:45.308477 16007 pending_rounds.cc:70] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Trying to abort 1 pending ops.
I20251023 02:10:45.308516 16007 pending_rounds.cc:77] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815583461818368 op_type: NO_OP noop_request { }
I20251023 02:10:45.308604 16007 raft_consensus.cc:2889] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:45.308682 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.309368 16007 tablet_replica.cc:333] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: stopping tablet replica
I20251023 02:10:45.309484 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.309607 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.310034 16007 tablet_replica.cc:333] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: stopping tablet replica
I20251023 02:10:45.310165 16007 raft_consensus.cc:2243] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.310267 16007 raft_consensus.cc:2272] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.310685 16007 tablet_replica.cc:333] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: stopping tablet replica
I20251023 02:10:45.310798 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.310904 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.325179 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
I20251023 02:10:45.328213 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:10:45.335438 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:45.335729 16007 tablet_replica.cc:333] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: stopping tablet replica
I20251023 02:10:45.335896 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.336000 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.336400 16007 tablet_replica.cc:333] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: stopping tablet replica
I20251023 02:10:45.336494 16007 raft_consensus.cc:2243] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.336577 16007 raft_consensus.cc:2272] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.336978 16007 tablet_replica.cc:333] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: stopping tablet replica
I20251023 02:10:45.337069 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.337147 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.337507 16007 tablet_replica.cc:333] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: stopping tablet replica
I20251023 02:10:45.337584 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.337661 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.351197 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:10:45.354108 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:10:45.361078 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:45.361337 16007 tablet_replica.cc:333] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: stopping tablet replica
I20251023 02:10:45.361547 16007 raft_consensus.cc:2243] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:45.361771 16007 pending_rounds.cc:70] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Trying to abort 1 pending ops.
I20251023 02:10:45.361812 16007 pending_rounds.cc:77] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815583500881920 op_type: NO_OP noop_request { }
I20251023 02:10:45.361908 16007 raft_consensus.cc:2889] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:45.361984 16007 raft_consensus.cc:2272] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.362648 16007 tablet_replica.cc:333] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: stopping tablet replica
I20251023 02:10:45.362782 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:45.362958 16007 pending_rounds.cc:70] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Trying to abort 1 pending ops.
I20251023 02:10:45.362991 16007 pending_rounds.cc:77] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815583145320448 op_type: NO_OP noop_request { }
I20251023 02:10:45.363068 16007 raft_consensus.cc:2889] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:45.363142 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.363695 16007 tablet_replica.cc:333] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: stopping tablet replica
I20251023 02:10:45.363807 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:45.363948 16007 pending_rounds.cc:70] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Trying to abort 1 pending ops.
I20251023 02:10:45.363981 16007 pending_rounds.cc:77] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815583873748992 op_type: NO_OP noop_request { }
I20251023 02:10:45.364048 16007 raft_consensus.cc:2889] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:45.364120 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.364638 16007 tablet_replica.cc:333] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: stopping tablet replica
I20251023 02:10:45.364753 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:45.364848 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:45.378559 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:10:45.381783 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:45.395474 22210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:45.395900 22209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:45.396328 16007 server_base.cc:1047] running on GCE node
W20251023 02:10:45.396617 22212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:45.396932 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:45.397022 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:45.397078 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185445397073 us; error 0 us; skew 500 ppm
I20251023 02:10:45.397219 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:45.398253 16007 webserver.cc:492] Webserver started at http://127.15.161.196:42019/ using document root <none> and password file <none>
I20251023 02:10:45.398447 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:45.398516 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:45.398645 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:45.399196 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "4eb08939483e457aafa743f7fdb9506e"
format_stamp: "Formatted at 2025-10-23 02:10:45 on dist-test-slave-bfbt"
I20251023 02:10:45.401297 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:45.402685 22217 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:45.403050 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:45.403153 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "4eb08939483e457aafa743f7fdb9506e"
format_stamp: "Formatted at 2025-10-23 02:10:45 on dist-test-slave-bfbt"
I20251023 02:10:45.403276 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:45.415582 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:45.415966 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:45.416379 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:45.417043 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:45.417107 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:45.417184 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:45.417229 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:45.428800 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:34973
I20251023 02:10:45.428830 22279 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:34973 every 8 connection(s)
I20251023 02:10:45.434314 22280 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:45.434444 22280 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:45.434679 22280 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:45.435355 21885 ts_manager.cc:194] Registered new tserver with Master: 4eb08939483e457aafa743f7fdb9506e (127.15.161.196:34973)
I20251023 02:10:45.435982 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60416
W20251023 02:10:45.757660 21933 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:45.758618 21872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20251023 02:10:45.759537 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:45.759627 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:45.760800 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:45.760866 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:45.761933 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:45.762006 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
I20251023 02:10:46.437005 22280 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
W20251023 02:10:46.765306 21933 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:46.767096 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:46.767156 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:46.768353 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:46.768414 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:46.769500 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:46.769567 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:47.772866 21933 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:47.774788 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:47.774875 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:47.776079 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:47.776145 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:47.777294 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:47.777365 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:48.780696 21933 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:48.782562 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:48.782632 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:48.783841 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:48.783901 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:48.784924 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:48.784981 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:49.788508 21933 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:49.790342 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:49.790429 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:49.791632 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:49.791694 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.195:45859: connect: Connection refused (error 111)
W20251023 02:10:49.792860 21933 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:49.792923 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.161.193:39333: connect: Connection refused (error 111)
W20251023 02:10:50.793712 21933 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 00b51eaa2b6142f7a0e665109a362654 not available for placement
I20251023 02:10:50.950453 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:50.953296 22298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:50.953364 22299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:50.953718 22301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:50.965754 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:50.966616 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:50.966702 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:50.966769 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185450966748 us; error 0 us; skew 500 ppm
I20251023 02:10:50.966972 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:50.968251 16007 webserver.cc:492] Webserver started at http://127.15.161.193:43935/ using document root <none> and password file <none>
I20251023 02:10:50.968472 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:50.968544 16007 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 02:10:50.970407 16007 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20251023 02:10:50.971872 22306 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:50.972218 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:50.972333 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "9165a251458341aebec6b4ece6f128e1"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:50.972460 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:50.997401 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:50.997864 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:50.998405 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:50.999704 22313 ts_tablet_manager.cc:548] Loading tablet metadata (0/4 complete)
I20251023 02:10:51.007576 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251023 02:10:51.007684 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20251023 02:10:51.007812 16007 ts_tablet_manager.cc:600] Registering tablets (0/4 complete)
I20251023 02:10:51.009709 22313 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:51.013103 22313 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.013480 22313 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Bootstrap complete.
I20251023 02:10:51.013707 22313 ts_tablet_manager.cc:1403] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:51.014266 22313 raft_consensus.cc:359] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:51.014412 22313 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.014501 16007 ts_tablet_manager.cc:616] Registered 4 tablets
I20251023 02:10:51.014590 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s user 0.007s sys 0.000s
I20251023 02:10:51.014655 22313 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:51.015398 22313 ts_tablet_manager.cc:1434] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:51.015623 22313 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:51.018743 22313 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.019065 22313 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Bootstrap complete.
I20251023 02:10:51.019327 22313 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:51.019821 22313 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.019948 22313 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.020119 22313 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.020440 22313 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:51.020612 22313 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:51.023823 22313 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:51.024296 22313 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Bootstrap complete.
I20251023 02:10:51.024542 22313 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:51.025126 22313 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.025329 22313 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.025559 22313 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.026034 22313 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251023 02:10:51.026322 22313 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Bootstrap starting.
I20251023 02:10:51.028929 22313 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.029218 22313 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Bootstrap complete.
I20251023 02:10:51.029492 22313 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251023 02:10:51.030084 22313 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.030297 22313 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.030526 22313 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.031047 22313 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:51.034662 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:39333
I20251023 02:10:51.034686 22374 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:39333 every 8 connection(s)
I20251023 02:10:51.035913 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:51.038763 22380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:51.038959 22379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:51.039438 22382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:51.040208 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:51.040480 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:51.040562 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:51.040628 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185451040602 us; error 0 us; skew 500 ppm
I20251023 02:10:51.040798 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:51.041893 16007 webserver.cc:492] Webserver started at http://127.15.161.194:33995/ using document root <none> and password file <none>
I20251023 02:10:51.042085 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:51.042194 16007 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 02:10:51.043839 16007 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:51.045063 22387 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:51.045334 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:51.045473 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "87e7fa17045341a4ba9533cada498b31"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:51.045663 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:51.046940 22375 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:51.047117 22375 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:51.047693 22375 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:51.049093 21885 ts_manager.cc:194] Re-registered known tserver with Master: 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333)
I20251023 02:10:51.050901 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60422
I20251023 02:10:51.053406 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:51.053864 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:51.054365 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:51.063719 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251023 02:10:51.063804 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20251023 02:10:51.065623 22394 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:51.068917 22394 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.069196 22394 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Bootstrap complete.
I20251023 02:10:51.069409 22394 ts_tablet_manager.cc:1403] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:51.069924 22394 raft_consensus.cc:359] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:51.070094 22394 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.070376 22394 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:51.070801 16007 ts_tablet_manager.cc:616] Registered 4 tablets
I20251023 02:10:51.070891 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s user 0.004s sys 0.004s
I20251023 02:10:51.071048 22394 ts_tablet_manager.cc:1434] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:51.071277 22394 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:51.074676 22394 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.075116 22394 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Bootstrap complete.
I20251023 02:10:51.075433 22394 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251023 02:10:51.076331 22394 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.076570 22394 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.076848 22394 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.077445 22394 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:10:51.077737 22394 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:51.082322 22394 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.082772 22394 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Bootstrap complete.
I20251023 02:10:51.083078 22394 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20251023 02:10:51.083853 22394 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.084055 22394 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.084309 22394 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.084700 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:39041
I20251023 02:10:51.084728 22454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:39041 every 8 connection(s)
I20251023 02:10:51.084854 22394 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 02:10:51.085059 22394 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Bootstrap starting.
I20251023 02:10:51.086299 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:51.089682 22458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:51.089929 22394 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.090327 22394 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Bootstrap complete.
I20251023 02:10:51.090756 22394 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
W20251023 02:10:51.091104 22460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:51.091795 22394 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.092027 22394 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.092340 22394 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.092957 22394 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 02:10:51.094960 16007 server_base.cc:1047] running on GCE node
W20251023 02:10:51.095351 22462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:51.095726 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:51.095809 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:51.095867 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185451095847 us; error 0 us; skew 500 ppm
I20251023 02:10:51.096024 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:51.097060 22455 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:51.097218 16007 webserver.cc:492] Webserver started at http://127.15.161.195:39229/ using document root <none> and password file <none>
I20251023 02:10:51.097225 22455 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:51.097419 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:51.097471 16007 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 02:10:51.097667 22455 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:51.098941 21885 ts_manager.cc:194] Re-registered known tserver with Master: 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:51.099421 16007 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 02:10:51.101274 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60428
I20251023 02:10:51.101409 22467 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:51.101778 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 02:10:51.101902 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "00b51eaa2b6142f7a0e665109a362654"
format_stamp: "Formatted at 2025-10-23 02:10:43 on dist-test-slave-bfbt"
I20251023 02:10:51.101995 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:51.102592 22455 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
I20251023 02:10:51.111208 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:51.111656 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:51.112075 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:51.120811 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251023 02:10:51.120903 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20251023 02:10:51.122706 22474 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:51.126711 22474 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:51.127012 16007 ts_tablet_manager.cc:616] Registered 4 tablets
I20251023 02:10:51.127080 22474 tablet_bootstrap.cc:492] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap complete.
I20251023 02:10:51.127085 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.005s sys 0.000s
I20251023 02:10:51.127358 22474 ts_tablet_manager.cc:1403] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 02:10:51.128093 22474 raft_consensus.cc:359] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:51.128306 22474 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.128547 22474 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:51.129102 22474 ts_tablet_manager.cc:1434] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:51.129314 22474 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:51.133281 22474 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:51.133630 22474 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap complete.
I20251023 02:10:51.133836 22474 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251023 02:10:51.134349 22474 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.134625 22474 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.134894 22474 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.135391 22474 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20251023 02:10:51.135571 22474 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:51.138788 22474 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:51.139178 22474 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Bootstrap complete.
I20251023 02:10:51.139415 22474 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:51.139988 22474 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.140121 22474 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.140326 22474 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.140794 22474 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:51.141072 22474 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap starting.
I20251023 02:10:51.144237 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:45859
I20251023 02:10:51.144237 22535 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:45859 every 8 connection(s)
I20251023 02:10:51.145006 22474 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:51.145514 22474 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Bootstrap complete.
I20251023 02:10:51.145792 22474 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251023 02:10:51.146538 22474 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.146824 22474 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Initialized, Role: FOLLOWER
I20251023 02:10:51.147066 22474 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:51.147512 22474 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 02:10:51.150830 22536 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34009
I20251023 02:10:51.150966 22536 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:51.151255 22536 heartbeater.cc:507] Master 127.15.161.254:34009 requested a full tablet report, sending...
I20251023 02:10:51.152298 21885 ts_manager.cc:194] Re-registered known tserver with Master: 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:51.153968 21885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60434
I20251023 02:10:51.154990 22536 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
W20251023 02:10:51.801875 21933 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Replica 9165a251458341aebec6b4ece6f128e1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } }
W20251023 02:10:51.803324 21933 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 1506a9a3315f483ca6869328f4cc0d0e, TS 00b51eaa2b6142f7a0e665109a362654 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
W20251023 02:10:51.803422 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 1506a9a3315f483ca6869328f4cc0d0e, TS 00b51eaa2b6142f7a0e665109a362654 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
W20251023 02:10:51.808852 21933 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 76bcdb4dce034db296f8ef2eafb633e5, TS 9165a251458341aebec6b4ece6f128e1 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
W20251023 02:10:51.808928 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 76bcdb4dce034db296f8ef2eafb633e5, TS 9165a251458341aebec6b4ece6f128e1 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
W20251023 02:10:51.809976 21933 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 3242b61802d64be189ad59b48a4aa5b8, TS 87e7fa17045341a4ba9533cada498b31 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
W20251023 02:10:51.810050 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 3242b61802d64be189ad59b48a4aa5b8, TS 87e7fa17045341a4ba9533cada498b31 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.052467 22375 heartbeater.cc:499] Master 127.15.161.254:34009 was elected leader, sending a full tablet report...
I20251023 02:10:52.231735 22542 raft_consensus.cc:493] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:52.231880 22542 raft_consensus.cc:515] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.232523 22542 leader_election.cc:290] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:52.238255 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:52.238363 22509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654" is_pre_election: true
I20251023 02:10:52.238499 22350 raft_consensus.cc:2410] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87e7fa17045341a4ba9533cada498b31 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 02:10:52.238559 22509 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e7fa17045341a4ba9533cada498b31 in term 1.
I20251023 02:10:52.238910 22391 leader_election.cc:304] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 87e7fa17045341a4ba9533cada498b31; no voters:
I20251023 02:10:52.239144 22542 raft_consensus.cc:2804] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:10:52.239220 22542 raft_consensus.cc:493] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:52.239293 22542 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.241031 22542 raft_consensus.cc:515] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.241508 22542 leader_election.cc:290] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:52.241892 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:52.241933 22509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654"
I20251023 02:10:52.242041 22350 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.242075 22509 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.244036 22509 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e7fa17045341a4ba9533cada498b31 in term 2.
I20251023 02:10:52.244208 22350 raft_consensus.cc:2410] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 87e7fa17045341a4ba9533cada498b31 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 02:10:52.244387 22391 leader_election.cc:304] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 87e7fa17045341a4ba9533cada498b31; no voters:
I20251023 02:10:52.244642 22542 raft_consensus.cc:2804] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:52.244966 22542 raft_consensus.cc:697] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 2 LEADER]: Becoming Leader. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Running, Role: LEADER
I20251023 02:10:52.245169 22542 consensus_queue.cc:237] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.247109 21885 catalog_manager.cc:5649] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 reported cstate change: term changed from 1 to 2, leader changed from 9165a251458341aebec6b4ece6f128e1 (127.15.161.193) to 87e7fa17045341a4ba9533cada498b31 (127.15.161.194). New cstate: current_term: 2 leader_uuid: "87e7fa17045341a4ba9533cada498b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:52.258807 22542 raft_consensus.cc:493] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:52.258935 22542 raft_consensus.cc:515] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.259310 22542 leader_election.cc:290] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859), 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333)
I20251023 02:10:52.259626 22509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654" is_pre_election: true
I20251023 02:10:52.259712 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:52.259816 22509 raft_consensus.cc:2410] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87e7fa17045341a4ba9533cada498b31 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 02:10:52.259873 22350 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e7fa17045341a4ba9533cada498b31 in term 1.
I20251023 02:10:52.260170 22391 leader_election.cc:304] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 87e7fa17045341a4ba9533cada498b31, 9165a251458341aebec6b4ece6f128e1; no voters: 00b51eaa2b6142f7a0e665109a362654
I20251023 02:10:52.260366 22542 raft_consensus.cc:2804] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:10:52.260457 22542 raft_consensus.cc:493] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:52.260543 22542 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.262071 22542 raft_consensus.cc:515] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.262588 22542 leader_election.cc:290] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 election: Requested vote from peers 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859), 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333)
I20251023 02:10:52.262943 22509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654"
I20251023 02:10:52.263002 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "87e7fa17045341a4ba9533cada498b31" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:52.263099 22509 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.263145 22350 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.264804 22509 raft_consensus.cc:2410] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 87e7fa17045341a4ba9533cada498b31 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 02:10:52.264802 22350 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e7fa17045341a4ba9533cada498b31 in term 2.
I20251023 02:10:52.265108 22391 leader_election.cc:304] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87e7fa17045341a4ba9533cada498b31, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:52.265309 22542 raft_consensus.cc:2804] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:52.265429 22542 raft_consensus.cc:697] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 2 LEADER]: Becoming Leader. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Running, Role: LEADER
I20251023 02:10:52.265635 22542 consensus_queue.cc:237] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.267380 21885 catalog_manager.cc:5649] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 reported cstate change: term changed from 1 to 2, leader changed from 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195) to 87e7fa17045341a4ba9533cada498b31 (127.15.161.194). New cstate: current_term: 2 leader_uuid: "87e7fa17045341a4ba9533cada498b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:52.284399 22552 raft_consensus.cc:493] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:52.284523 22552 raft_consensus.cc:515] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.285110 22552 leader_election.cc:290] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:52.291060 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:52.291204 22350 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:52.291535 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:52.291563 22471 leader_election.cc:304] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:52.291687 22429 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:52.291796 22552 raft_consensus.cc:2804] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:10:52.291887 22552 raft_consensus.cc:493] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:52.291983 22552 raft_consensus.cc:3060] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.293664 22552 raft_consensus.cc:515] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.294142 22552 leader_election.cc:290] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:52.294421 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:52.294570 22350 raft_consensus.cc:3060] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.294538 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d720c77c125e449dab6cfbe71b57c196" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:52.294716 22429 raft_consensus.cc:3060] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.296686 22350 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 2.
I20251023 02:10:52.296868 22429 raft_consensus.cc:2468] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 2.
I20251023 02:10:52.296983 22471 leader_election.cc:304] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:52.297204 22552 raft_consensus.cc:2804] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:52.297541 22552 raft_consensus.cc:697] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 2 LEADER]: Becoming Leader. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Running, Role: LEADER
I20251023 02:10:52.297725 22552 consensus_queue.cc:237] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.299396 21885 catalog_manager.cc:5649] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:52.464421 22552 raft_consensus.cc:493] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:52.464586 22552 raft_consensus.cc:515] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.465139 22552 leader_election.cc:290] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:52.465519 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:52.465528 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:52.465708 22350 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:52.465730 22429 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 1.
I20251023 02:10:52.466034 22471 leader_election.cc:304] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:52.466271 22552 raft_consensus.cc:2804] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:10:52.466352 22552 raft_consensus.cc:493] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:52.466450 22552 raft_consensus.cc:3060] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.468361 22552 raft_consensus.cc:515] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.468859 22552 leader_election.cc:290] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:52.469285 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:52.469286 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3242b61802d64be189ad59b48a4aa5b8" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:52.469484 22350 raft_consensus.cc:3060] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.469491 22429 raft_consensus.cc:3060] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:52.472308 22429 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 2.
I20251023 02:10:52.472308 22350 raft_consensus.cc:2468] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 2.
I20251023 02:10:52.472703 22471 leader_election.cc:304] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 87e7fa17045341a4ba9533cada498b31, 9165a251458341aebec6b4ece6f128e1; no voters:
I20251023 02:10:52.472954 22552 raft_consensus.cc:2804] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:52.473107 22552 raft_consensus.cc:697] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 2 LEADER]: Becoming Leader. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Running, Role: LEADER
I20251023 02:10:52.473358 22552 consensus_queue.cc:237] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.475454 21885 catalog_manager.cc:5649] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:52.494908 22552 raft_consensus.cc:493] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:52.495034 22552 raft_consensus.cc:515] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.495419 22552 leader_election.cc:290] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:52.495803 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:52.495800 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1" is_pre_election: true
I20251023 02:10:52.496021 22350 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 2.
I20251023 02:10:52.496361 22471 leader_election.cc:304] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters: 87e7fa17045341a4ba9533cada498b31
I20251023 02:10:52.496553 22552 raft_consensus.cc:2804] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 02:10:52.496624 22552 raft_consensus.cc:493] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:52.496722 22552 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:52.498308 22552 raft_consensus.cc:515] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.498776 22552 leader_election.cc:290] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 3 election: Requested vote from peers 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333), 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041)
I20251023 02:10:52.499148 22350 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9165a251458341aebec6b4ece6f128e1"
I20251023 02:10:52.499145 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76bcdb4dce034db296f8ef2eafb633e5" candidate_uuid: "00b51eaa2b6142f7a0e665109a362654" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:52.499306 22350 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:52.500967 22350 raft_consensus.cc:2468] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b51eaa2b6142f7a0e665109a362654 in term 3.
I20251023 02:10:52.501273 22471 leader_election.cc:304] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters: 87e7fa17045341a4ba9533cada498b31
I20251023 02:10:52.501485 22552 raft_consensus.cc:2804] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:10:52.501605 22552 raft_consensus.cc:697] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 3 LEADER]: Becoming Leader. State: Replica: 00b51eaa2b6142f7a0e665109a362654, State: Running, Role: LEADER
I20251023 02:10:52.501818 22552 consensus_queue.cc:237] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.503788 21885 catalog_manager.cc:5649] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: term changed from 2 to 3, leader changed from 87e7fa17045341a4ba9533cada498b31 (127.15.161.194) to 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195). New cstate: current_term: 3 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:52.546762 22560 raft_consensus.cc:493] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:52.546883 22560 raft_consensus.cc:515] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.547498 22560 leader_election.cc:290] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041), 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:52.552865 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31" is_pre_election: true
I20251023 02:10:52.553086 22509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654" is_pre_election: true
I20251023 02:10:52.553256 22509 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9165a251458341aebec6b4ece6f128e1 in term 2.
I20251023 02:10:52.553592 22310 leader_election.cc:304] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters: 87e7fa17045341a4ba9533cada498b31
I20251023 02:10:52.553802 22560 raft_consensus.cc:2804] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 02:10:52.553869 22560 raft_consensus.cc:493] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:52.553947 22560 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:52.555521 22560 raft_consensus.cc:515] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.555964 22560 leader_election.cc:290] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 3 election: Requested vote from peers 87e7fa17045341a4ba9533cada498b31 (127.15.161.194:39041), 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:52.556221 22429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "87e7fa17045341a4ba9533cada498b31"
I20251023 02:10:52.556299 22509 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1506a9a3315f483ca6869328f4cc0d0e" candidate_uuid: "9165a251458341aebec6b4ece6f128e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "00b51eaa2b6142f7a0e665109a362654"
I20251023 02:10:52.556455 22509 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:52.558070 22509 raft_consensus.cc:2468] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9165a251458341aebec6b4ece6f128e1 in term 3.
I20251023 02:10:52.558385 22310 leader_election.cc:304] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00b51eaa2b6142f7a0e665109a362654, 9165a251458341aebec6b4ece6f128e1; no voters: 87e7fa17045341a4ba9533cada498b31
I20251023 02:10:52.558607 22560 raft_consensus.cc:2804] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 02:10:52.558983 22560 raft_consensus.cc:697] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 3 LEADER]: Becoming Leader. State: Replica: 9165a251458341aebec6b4ece6f128e1, State: Running, Role: LEADER
I20251023 02:10:52.559264 22560 consensus_queue.cc:237] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.561110 21885 catalog_manager.cc:5649] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 reported cstate change: term changed from 2 to 3, leader changed from 87e7fa17045341a4ba9533cada498b31 (127.15.161.194) to 9165a251458341aebec6b4ece6f128e1 (127.15.161.193). New cstate: current_term: 3 leader_uuid: "9165a251458341aebec6b4ece6f128e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:52.653296 22509 raft_consensus.cc:1240] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Rejecting Update request from peer 87e7fa17045341a4ba9533cada498b31 for earlier term 2. Current term is 3. Ops: []
I20251023 02:10:52.653820 22542 consensus_queue.cc:1059] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.654064 22545 raft_consensus.cc:3055] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:10:52.654193 22545 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Running, Role: LEADER
I20251023 02:10:52.654399 22545 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.654623 22545 raft_consensus.cc:3060] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Advancing to term 3
I20251023 02:10:52.699208 22509 raft_consensus.cc:1240] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 3 LEADER]: Rejecting Update request from peer 87e7fa17045341a4ba9533cada498b31 for earlier term 2. Current term is 3. Ops: []
I20251023 02:10:52.699689 22545 consensus_queue.cc:1059] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.699925 22545 consensus_queue.cc:1243] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [LEADER]: Peer 00b51eaa2b6142f7a0e665109a362654 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 02:10:52.700057 22542 raft_consensus.cc:3055] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 02:10:52.700145 22542 raft_consensus.cc:740] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 87e7fa17045341a4ba9533cada498b31, State: Running, Role: LEADER
I20251023 02:10:52.700376 22542 consensus_queue.cc:260] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
I20251023 02:10:52.716542 22542 raft_consensus.cc:3060] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Advancing to term 3
W20251023 02:10:52.813220 21933 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251023 02:10:52.814424 21933 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 76bcdb4dce034db296f8ef2eafb633e5, TS 9165a251458341aebec6b4ece6f128e1 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
W20251023 02:10:52.814479 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 76bcdb4dce034db296f8ef2eafb633e5, TS 9165a251458341aebec6b4ece6f128e1 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } }
W20251023 02:10:52.815532 21933 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 1506a9a3315f483ca6869328f4cc0d0e, TS 00b51eaa2b6142f7a0e665109a362654 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
W20251023 02:10:52.815583 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 1506a9a3315f483ca6869328f4cc0d0e, TS 00b51eaa2b6142f7a0e665109a362654 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
W20251023 02:10:52.816542 21933 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 3242b61802d64be189ad59b48a4aa5b8, TS 87e7fa17045341a4ba9533cada498b31 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
W20251023 02:10:52.816591 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 3242b61802d64be189ad59b48a4aa5b8, TS 87e7fa17045341a4ba9533cada498b31 -> TS 4eb08939483e457aafa743f7fdb9506e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } }
I20251023 02:10:52.849344 22350 raft_consensus.cc:1275] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:52.849911 22552 consensus_queue.cc:1048] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.853735 22429 raft_consensus.cc:1275] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:52.854317 22552 consensus_queue.cc:1048] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.975358 22429 raft_consensus.cc:1275] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 02:10:52.975756 22552 consensus_queue.cc:1048] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.975939 22552 consensus_queue.cc:1243] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Peer 87e7fa17045341a4ba9533cada498b31 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 02:10:52.976649 22429 pending_rounds.cc:85] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Aborting all ops after (but not including) 0
I20251023 02:10:52.976716 22429 pending_rounds.cc:107] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251023 02:10:52.976756 22429 raft_consensus.cc:2889] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 02:10:52.978566 22428 raft_consensus.cc:1275] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Refusing update from remote peer 9165a251458341aebec6b4ece6f128e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 02:10:52.979020 22560 consensus_queue.cc:1048] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.979187 22560 consensus_queue.cc:1243] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Peer 87e7fa17045341a4ba9533cada498b31 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 02:10:52.979496 22350 raft_consensus.cc:1275] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 02:10:52.980015 22570 consensus_queue.cc:1048] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:52.980198 22429 pending_rounds.cc:85] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Aborting all ops after (but not including) 0
I20251023 02:10:52.980301 22429 pending_rounds.cc:107] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251023 02:10:52.980386 22429 raft_consensus.cc:2889] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 02:10:52.982899 22509 raft_consensus.cc:1275] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Refusing update from remote peer 9165a251458341aebec6b4ece6f128e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 02:10:52.983367 22563 consensus_queue.cc:1048] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:53.025799 22429 raft_consensus.cc:1275] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:53.026276 22570 consensus_queue.cc:1048] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:53.029413 22350 raft_consensus.cc:1275] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 02:10:53.029989 22575 consensus_queue.cc:1048] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:53.820240 22509 consensus_queue.cc:237] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:53.822160 22350 raft_consensus.cc:1275] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:10:53.822305 22429 raft_consensus.cc:1275] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:10:53.822885 22575 consensus_queue.cc:1048] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:53.823148 22570 consensus_queue.cc:1048] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:53.825151 22556 raft_consensus.cc:2955] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.825961 22429 raft_consensus.cc:2955] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.825721 22350 raft_consensus.cc:2955] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
W20251023 02:10:53.829972 22468 consensus_peers.cc:597] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 -> Peer 4eb08939483e457aafa743f7fdb9506e (127.15.161.196:34973): Couldn't send request to peer 4eb08939483e457aafa743f7fdb9506e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d720c77c125e449dab6cfbe71b57c196. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:53.830791 21885 catalog_manager.cc:5649] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New cstate: current_term: 2 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:53.832356 22348 consensus_queue.cc:237] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:53.834671 22509 raft_consensus.cc:1275] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Refusing update from remote peer 9165a251458341aebec6b4ece6f128e1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 02:10:53.834849 22429 raft_consensus.cc:1275] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Refusing update from remote peer 9165a251458341aebec6b4ece6f128e1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 02:10:53.835295 22560 consensus_queue.cc:1048] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:53.835512 22563 consensus_queue.cc:1048] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:53.837911 22563 raft_consensus.cc:2955] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.838366 22509 raft_consensus.cc:2955] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
W20251023 02:10:53.841892 22307 consensus_peers.cc:597] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 -> Peer 4eb08939483e457aafa743f7fdb9506e (127.15.161.196:34973): Couldn't send request to peer 4eb08939483e457aafa743f7fdb9506e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1506a9a3315f483ca6869328f4cc0d0e. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:53.842164 22429 raft_consensus.cc:2955] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.843155 21883 catalog_manager.cc:5649] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New cstate: current_term: 3 leader_uuid: "9165a251458341aebec6b4ece6f128e1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.844594 22509 consensus_queue.cc:237] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
W20251023 02:10:53.847769 22468 consensus_peers.cc:597] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 -> Peer 4eb08939483e457aafa743f7fdb9506e (127.15.161.196:34973): Couldn't send request to peer 4eb08939483e457aafa743f7fdb9506e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3242b61802d64be189ad59b48a4aa5b8. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:53.847806 22350 raft_consensus.cc:1275] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:10:53.848140 22575 consensus_queue.cc:1048] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:53.848243 22429 raft_consensus.cc:1275] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Refusing update from remote peer 00b51eaa2b6142f7a0e665109a362654: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 02:10:53.848623 22570 consensus_queue.cc:1048] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:53.849825 22575 raft_consensus.cc:2955] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.850607 22429 raft_consensus.cc:2955] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.852991 22350 raft_consensus.cc:2955] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } } }
I20251023 02:10:53.853614 21883 catalog_manager.cc:5649] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 reported cstate change: config changed from index -1 to 3, NON_VOTER 4eb08939483e457aafa743f7fdb9506e (127.15.161.196) added. New cstate: current_term: 2 leader_uuid: "00b51eaa2b6142f7a0e665109a362654" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:54.335762 22585 ts_tablet_manager.cc:933] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: Initiating tablet copy from peer 9165a251458341aebec6b4ece6f128e1 (127.15.161.193:39333)
I20251023 02:10:54.336256 22585 tablet_copy_client.cc:323] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.193:39333
I20251023 02:10:54.344326 22360 tablet_copy_service.cc:140] P 9165a251458341aebec6b4ece6f128e1: Received BeginTabletCopySession request for tablet 1506a9a3315f483ca6869328f4cc0d0e from peer 4eb08939483e457aafa743f7fdb9506e ({username='slave'} at 127.0.0.1:51104)
I20251023 02:10:54.344455 22360 tablet_copy_service.cc:161] P 9165a251458341aebec6b4ece6f128e1: Beginning new tablet copy session on tablet 1506a9a3315f483ca6869328f4cc0d0e from peer 4eb08939483e457aafa743f7fdb9506e at {username='slave'} at 127.0.0.1:51104: session id = 4eb08939483e457aafa743f7fdb9506e-1506a9a3315f483ca6869328f4cc0d0e
I20251023 02:10:54.346589 22360 tablet_copy_source_session.cc:215] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:54.347509 22585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506a9a3315f483ca6869328f4cc0d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:54.351867 22585 tablet_copy_client.cc:806] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:54.352075 22585 tablet_copy_client.cc:670] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:54.353775 22585 tablet_copy_client.cc:538] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:54.356240 22585 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: Bootstrap starting.
I20251023 02:10:54.360845 22585 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:54.361086 22585 tablet_bootstrap.cc:492] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: Bootstrap complete.
I20251023 02:10:54.361258 22585 ts_tablet_manager.cc:1403] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20251023 02:10:54.361860 22585 raft_consensus.cc:359] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:54.361990 22585 raft_consensus.cc:740] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 4eb08939483e457aafa743f7fdb9506e, State: Initialized, Role: LEARNER
I20251023 02:10:54.362170 22585 consensus_queue.cc:260] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } attrs { replace: true } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:54.362804 22585 ts_tablet_manager.cc:1434] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251023 02:10:54.363323 22360 tablet_copy_service.cc:342] P 9165a251458341aebec6b4ece6f128e1: Request end of tablet copy session 4eb08939483e457aafa743f7fdb9506e-1506a9a3315f483ca6869328f4cc0d0e received from {username='slave'} at 127.0.0.1:51104
I20251023 02:10:54.363430 22360 tablet_copy_service.cc:434] P 9165a251458341aebec6b4ece6f128e1: ending tablet copy session 4eb08939483e457aafa743f7fdb9506e-1506a9a3315f483ca6869328f4cc0d0e on tablet 1506a9a3315f483ca6869328f4cc0d0e with peer 4eb08939483e457aafa743f7fdb9506e
I20251023 02:10:54.376339 22585 ts_tablet_manager.cc:933] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: Initiating tablet copy from peer 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:54.376911 22585 tablet_copy_client.cc:323] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:45859
I20251023 02:10:54.381812 22520 tablet_copy_service.cc:140] P 00b51eaa2b6142f7a0e665109a362654: Received BeginTabletCopySession request for tablet 3242b61802d64be189ad59b48a4aa5b8 from peer 4eb08939483e457aafa743f7fdb9506e ({username='slave'} at 127.0.0.1:57514)
I20251023 02:10:54.381958 22520 tablet_copy_service.cc:161] P 00b51eaa2b6142f7a0e665109a362654: Beginning new tablet copy session on tablet 3242b61802d64be189ad59b48a4aa5b8 from peer 4eb08939483e457aafa743f7fdb9506e at {username='slave'} at 127.0.0.1:57514: session id = 4eb08939483e457aafa743f7fdb9506e-3242b61802d64be189ad59b48a4aa5b8
I20251023 02:10:54.384150 22520 tablet_copy_source_session.cc:215] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:54.384923 22585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3242b61802d64be189ad59b48a4aa5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:54.388492 22585 tablet_copy_client.cc:806] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:54.388701 22585 tablet_copy_client.cc:670] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:54.389838 22585 tablet_copy_client.cc:538] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:54.390754 22593 ts_tablet_manager.cc:933] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: Initiating tablet copy from peer 00b51eaa2b6142f7a0e665109a362654 (127.15.161.195:45859)
I20251023 02:10:54.391414 22593 tablet_copy_client.cc:323] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:45859
I20251023 02:10:54.391947 22520 tablet_copy_service.cc:140] P 00b51eaa2b6142f7a0e665109a362654: Received BeginTabletCopySession request for tablet d720c77c125e449dab6cfbe71b57c196 from peer 4eb08939483e457aafa743f7fdb9506e ({username='slave'} at 127.0.0.1:57514)
I20251023 02:10:54.392071 22520 tablet_copy_service.cc:161] P 00b51eaa2b6142f7a0e665109a362654: Beginning new tablet copy session on tablet d720c77c125e449dab6cfbe71b57c196 from peer 4eb08939483e457aafa743f7fdb9506e at {username='slave'} at 127.0.0.1:57514: session id = 4eb08939483e457aafa743f7fdb9506e-d720c77c125e449dab6cfbe71b57c196
I20251023 02:10:54.393128 22585 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: Bootstrap starting.
I20251023 02:10:54.394354 22520 tablet_copy_source_session.cc:215] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:54.395076 22593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d720c77c125e449dab6cfbe71b57c196. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:54.398993 22585 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:54.399330 22585 tablet_bootstrap.cc:492] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: Bootstrap complete.
I20251023 02:10:54.399596 22585 ts_tablet_manager.cc:1403] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: Time spent bootstrapping tablet: real 0.007s user 0.008s sys 0.001s
I20251023 02:10:54.399900 22593 tablet_copy_client.cc:806] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:54.400094 22593 tablet_copy_client.cc:670] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:54.400731 22585 raft_consensus.cc:359] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:54.400916 22585 raft_consensus.cc:740] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4eb08939483e457aafa743f7fdb9506e, State: Initialized, Role: LEARNER
I20251023 02:10:54.401103 22585 consensus_queue.cc:260] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } attrs { replace: true } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:54.401189 22593 tablet_copy_client.cc:538] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:54.401854 22585 ts_tablet_manager.cc:1434] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:54.402405 22520 tablet_copy_service.cc:342] P 00b51eaa2b6142f7a0e665109a362654: Request end of tablet copy session 4eb08939483e457aafa743f7fdb9506e-3242b61802d64be189ad59b48a4aa5b8 received from {username='slave'} at 127.0.0.1:57514
I20251023 02:10:54.402552 22520 tablet_copy_service.cc:434] P 00b51eaa2b6142f7a0e665109a362654: ending tablet copy session 4eb08939483e457aafa743f7fdb9506e-3242b61802d64be189ad59b48a4aa5b8 on tablet 3242b61802d64be189ad59b48a4aa5b8 with peer 4eb08939483e457aafa743f7fdb9506e
I20251023 02:10:54.404592 22593 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: Bootstrap starting.
I20251023 02:10:54.409468 22593 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:54.409780 22593 tablet_bootstrap.cc:492] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: Bootstrap complete.
I20251023 02:10:54.410040 22593 ts_tablet_manager.cc:1403] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20251023 02:10:54.410702 22593 raft_consensus.cc:359] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:54.410848 22593 raft_consensus.cc:740] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4eb08939483e457aafa743f7fdb9506e, State: Initialized, Role: LEARNER
I20251023 02:10:54.411027 22593 consensus_queue.cc:260] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "9165a251458341aebec6b4ece6f128e1" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 39333 } attrs { replace: true } } peers { permanent_uuid: "87e7fa17045341a4ba9533cada498b31" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 39041 } } peers { permanent_uuid: "00b51eaa2b6142f7a0e665109a362654" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 45859 } } peers { permanent_uuid: "4eb08939483e457aafa743f7fdb9506e" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 34973 } attrs { promote: true } }
I20251023 02:10:54.411520 22593 ts_tablet_manager.cc:1434] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:54.412127 22520 tablet_copy_service.cc:342] P 00b51eaa2b6142f7a0e665109a362654: Request end of tablet copy session 4eb08939483e457aafa743f7fdb9506e-d720c77c125e449dab6cfbe71b57c196 received from {username='slave'} at 127.0.0.1:57514
I20251023 02:10:54.412250 22520 tablet_copy_service.cc:434] P 00b51eaa2b6142f7a0e665109a362654: ending tablet copy session 4eb08939483e457aafa743f7fdb9506e-d720c77c125e449dab6cfbe71b57c196 on tablet d720c77c125e449dab6cfbe71b57c196 with peer 4eb08939483e457aafa743f7fdb9506e
I20251023 02:10:54.894989 22255 raft_consensus.cc:1217] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251023 02:10:54.922109 22255 raft_consensus.cc:1217] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251023 02:10:55.000056 22255 raft_consensus.cc:1217] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251023 02:10:55.171612 16007 tablet_server.cc:178] TabletServer@127.15.161.193:39333 shutting down...
W20251023 02:10:55.173691 21932 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20251023 02:10:55.175336 21933 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:10:55.175422 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 02:10:55.179196 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:55.179414 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.179571 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.179744 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.180276 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.180414 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 3 LEADER]: Raft consensus shutting down.
I20251023 02:10:55.180768 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.181192 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.181319 16007 raft_consensus.cc:2243] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.181465 16007 raft_consensus.cc:2272] T 76bcdb4dce034db296f8ef2eafb633e5 P 9165a251458341aebec6b4ece6f128e1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.181883 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.182015 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.182188 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 9165a251458341aebec6b4ece6f128e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.197254 16007 tablet_server.cc:195] TabletServer@127.15.161.193:39333 shutdown complete.
I20251023 02:10:55.201246 16007 tablet_server.cc:178] TabletServer@127.15.161.194:39041 shutting down...
I20251023 02:10:55.210517 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:55.210716 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.210877 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.211071 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.211506 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.211647 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.211835 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.212301 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.212437 16007 raft_consensus.cc:2243] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.212565 16007 raft_consensus.cc:2272] T 76bcdb4dce034db296f8ef2eafb633e5 P 87e7fa17045341a4ba9533cada498b31 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.213054 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.213192 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.213364 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 87e7fa17045341a4ba9533cada498b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.235219 16007 tablet_server.cc:195] TabletServer@127.15.161.194:39041 shutdown complete.
I20251023 02:10:55.238925 16007 tablet_server.cc:178] TabletServer@127.15.161.195:45859 shutting down...
W20251023 02:10:55.240622 21933 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:10:55.240703 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:10:55.242391 21933 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 02:10:55.242470 21933 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 02:10:55.247530 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:55.247704 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.247849 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:10:55.248134 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.248526 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.248641 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:55.248795 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.249150 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.249254 16007 raft_consensus.cc:2243] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 3 LEADER]: Raft consensus shutting down.
I20251023 02:10:55.249449 16007 raft_consensus.cc:2272] T 76bcdb4dce034db296f8ef2eafb633e5 P 00b51eaa2b6142f7a0e665109a362654 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.249816 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:55.249918 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:10:55.250214 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 00b51eaa2b6142f7a0e665109a362654 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.253931 16007 tablet_server.cc:195] TabletServer@127.15.161.195:45859 shutdown complete.
I20251023 02:10:55.256803 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:10:55.263768 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:55.263969 16007 tablet_replica.cc:333] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e: stopping tablet replica
I20251023 02:10:55.264117 16007 raft_consensus.cc:2243] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Raft consensus shutting down.
I20251023 02:10:55.264216 16007 raft_consensus.cc:2272] T d720c77c125e449dab6cfbe71b57c196 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Raft consensus is shut down!
I20251023 02:10:55.264549 16007 tablet_replica.cc:333] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e: stopping tablet replica
I20251023 02:10:55.264632 16007 raft_consensus.cc:2243] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e [term 3 LEARNER]: Raft consensus shutting down.
I20251023 02:10:55.264711 16007 raft_consensus.cc:2272] T 1506a9a3315f483ca6869328f4cc0d0e P 4eb08939483e457aafa743f7fdb9506e [term 3 LEARNER]: Raft consensus is shut down!
I20251023 02:10:55.265018 16007 tablet_replica.cc:333] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e: stopping tablet replica
I20251023 02:10:55.265101 16007 raft_consensus.cc:2243] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Raft consensus shutting down.
I20251023 02:10:55.265180 16007 raft_consensus.cc:2272] T 3242b61802d64be189ad59b48a4aa5b8 P 4eb08939483e457aafa743f7fdb9506e [term 2 LEARNER]: Raft consensus is shut down!
I20251023 02:10:55.278102 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:10:55.280691 16007 master.cc:561] Master@127.15.161.254:34009 shutting down...
I20251023 02:10:55.287108 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:55.287271 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:55.287349 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P fd187014ebc04229bef4615b248db85a: stopping tablet replica
I20251023 02:10:55.299944 16007 master.cc:583] Master@127.15.161.254:34009 shutdown complete.
[ OK ] AutoRebalancerTest.TestHandlingFailedTservers (11612 ms)
[ RUN ] AutoRebalancerTest.TestDeletedTables
I20251023 02:10:55.319406 16007 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.161.254:34399
I20251023 02:10:55.319814 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:55.321687 22602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.321803 22603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.322026 22605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:55.322330 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:55.322515 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:55.322558 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:55.322589 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185455322585 us; error 0 us; skew 500 ppm
I20251023 02:10:55.322741 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:55.323741 16007 webserver.cc:492] Webserver started at http://127.15.161.254:42419/ using document root <none> and password file <none>
I20251023 02:10:55.323940 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:55.323992 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:55.324090 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:55.324606 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/master-0-root/instance:
uuid: "4675b1f40eb94dce84c3f101d95b6e8e"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.326927 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:55.328312 22610 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.328605 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:55.328681 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/master-0-root
uuid: "4675b1f40eb94dce84c3f101d95b6e8e"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.328763 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:55.335991 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:55.336331 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:55.345428 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.254:34399
I20251023 02:10:55.345445 22661 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.254:34399 every 8 connection(s)
I20251023 02:10:55.346515 22662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:55.350276 22662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Bootstrap starting.
I20251023 02:10:55.351233 22662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:55.352406 22662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: No bootstrap required, opened a new log
I20251023 02:10:55.352936 22662 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" member_type: VOTER }
I20251023 02:10:55.353045 22662 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:55.353086 22662 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4675b1f40eb94dce84c3f101d95b6e8e, State: Initialized, Role: FOLLOWER
I20251023 02:10:55.353215 22662 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" member_type: VOTER }
I20251023 02:10:55.353299 22662 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 02:10:55.353336 22662 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 02:10:55.353384 22662 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:55.355026 22662 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" member_type: VOTER }
I20251023 02:10:55.355162 22662 leader_election.cc:304] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4675b1f40eb94dce84c3f101d95b6e8e; no voters:
I20251023 02:10:55.355428 22662 leader_election.cc:290] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 02:10:55.355521 22665 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:55.355960 22665 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 1 LEADER]: Becoming Leader. State: Replica: 4675b1f40eb94dce84c3f101d95b6e8e, State: Running, Role: LEADER
I20251023 02:10:55.356141 22662 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [sys.catalog]: configured and running, proceeding with master startup.
I20251023 02:10:55.356138 22665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" member_type: VOTER }
I20251023 02:10:55.356884 22667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4675b1f40eb94dce84c3f101d95b6e8e. Latest consensus state: current_term: 1 leader_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" member_type: VOTER } }
I20251023 02:10:55.357100 22667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:55.356933 22666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4675b1f40eb94dce84c3f101d95b6e8e" member_type: VOTER } }
I20251023 02:10:55.357483 22666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [sys.catalog]: This master's current role is: LEADER
I20251023 02:10:55.358079 22671 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 02:10:55.359179 22671 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 02:10:55.362198 22671 catalog_manager.cc:1357] Generated new cluster ID: 7884038d310642428ad9d916c91873cf
I20251023 02:10:55.362273 22671 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 02:10:55.362821 16007 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 02:10:55.390493 22671 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 02:10:55.391088 22671 catalog_manager.cc:1514] Loading token signing keys...
I20251023 02:10:55.400475 22671 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Generated new TSK 0
I20251023 02:10:55.400635 22671 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 02:10:55.427557 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:55.429677 22683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.429780 22684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.430218 22686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:55.430665 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:55.430832 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:55.430876 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:55.430902 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185455430900 us; error 0 us; skew 500 ppm
I20251023 02:10:55.430989 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:55.432004 16007 webserver.cc:492] Webserver started at http://127.15.161.193:39345/ using document root <none> and password file <none>
I20251023 02:10:55.432178 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:55.432225 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:55.432307 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:55.432843 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/instance:
uuid: "7fb388975cb842e7b476da64bd50a992"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.435148 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:55.436412 22691 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.436703 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:55.436820 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "7fb388975cb842e7b476da64bd50a992"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.436901 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:55.448463 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:55.448876 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:55.449328 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:55.450054 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:55.450103 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.450177 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:55.450217 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.461869 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:38447
I20251023 02:10:55.461900 22753 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:38447 every 8 connection(s)
I20251023 02:10:55.463284 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:55.465456 22758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.465642 22759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.466156 22761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:55.471083 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:55.471280 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:55.471324 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:55.471345 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185455471342 us; error 0 us; skew 500 ppm
I20251023 02:10:55.471439 22754 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34399
I20251023 02:10:55.471464 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:55.471536 22754 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:55.471885 22754 heartbeater.cc:507] Master 127.15.161.254:34399 requested a full tablet report, sending...
I20251023 02:10:55.472535 16007 webserver.cc:492] Webserver started at http://127.15.161.194:34347/ using document root <none> and password file <none>
I20251023 02:10:55.472808 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:55.472824 22627 ts_manager.cc:194] Registered new tserver with Master: 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447)
I20251023 02:10:55.472911 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:55.473032 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:55.473713 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/instance:
uuid: "6e35508ce4004db383775ecd53a92ba0"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.474258 22627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53366
I20251023 02:10:55.475992 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 02:10:55.477201 22766 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.477478 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:55.477555 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root
uuid: "6e35508ce4004db383775ecd53a92ba0"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.477632 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:55.488817 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:55.489131 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:55.489506 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:55.490082 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:55.490149 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.490214 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:55.490252 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.500449 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.194:35611
I20251023 02:10:55.500468 22828 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.194:35611 every 8 connection(s)
I20251023 02:10:55.501873 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:55.504449 22832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.504537 22833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:55.505926 22835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:55.506016 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:55.506578 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:55.506655 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:55.506687 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185455506683 us; error 0 us; skew 500 ppm
I20251023 02:10:55.506834 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:55.507779 22829 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34399
I20251023 02:10:55.507886 22829 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:55.508059 16007 webserver.cc:492] Webserver started at http://127.15.161.195:41803/ using document root <none> and password file <none>
I20251023 02:10:55.508232 22829 heartbeater.cc:507] Master 127.15.161.254:34399 requested a full tablet report, sending...
I20251023 02:10:55.508260 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:55.508318 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:55.508447 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:55.508960 22627 ts_manager.cc:194] Registered new tserver with Master: 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:55.509084 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/instance:
uuid: "09b43b6f0a944b9096101927ba53a14b"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.509754 22627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53376
I20251023 02:10:55.511441 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 02:10:55.512665 22840 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.512998 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:55.513088 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root
uuid: "09b43b6f0a944b9096101927ba53a14b"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:55.513176 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:55.518246 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:55.518610 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:55.518996 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:55.519620 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:55.519668 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.519721 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:55.519753 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:55.529852 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.195:34867
I20251023 02:10:55.529879 22902 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.195:34867 every 8 connection(s)
I20251023 02:10:55.535053 22903 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34399
I20251023 02:10:55.535133 22903 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:55.535391 22903 heartbeater.cc:507] Master 127.15.161.254:34399 requested a full tablet report, sending...
I20251023 02:10:55.536024 22627 ts_manager.cc:194] Registered new tserver with Master: 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:55.536195 16007 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005564009s
I20251023 02:10:55.536628 22627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53380
I20251023 02:10:56.475458 22754 heartbeater.cc:499] Master 127.15.161.254:34399 was elected leader, sending a full tablet report...
I20251023 02:10:56.510783 22829 heartbeater.cc:499] Master 127.15.161.254:34399 was elected leader, sending a full tablet report...
I20251023 02:10:56.537621 22903 heartbeater.cc:499] Master 127.15.161.254:34399 was elected leader, sending a full tablet report...
I20251023 02:10:56.563156 16007 test_util.cc:276] Using random seed: -242900016
I20251023 02:10:56.577723 22627 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53396:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:10:56.578322 22627 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:10:56.597934 22719 tablet_service.cc:1505] Processing CreateTablet for tablet cea9618bd841449b8db98622d5a12e2d (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:56.598273 22718 tablet_service.cc:1505] Processing CreateTablet for tablet 75d3ebdb6f6e49829e45d892bb203c05 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:56.598462 22719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea9618bd841449b8db98622d5a12e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.598661 22718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75d3ebdb6f6e49829e45d892bb203c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.602447 22717 tablet_service.cc:1505] Processing CreateTablet for tablet 6b66d971101b4acdb207b7e7e36799d4 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:56.602864 22717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b66d971101b4acdb207b7e7e36799d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.603380 22794 tablet_service.cc:1505] Processing CreateTablet for tablet cea9618bd841449b8db98622d5a12e2d (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:56.603801 22794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea9618bd841449b8db98622d5a12e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.604116 22792 tablet_service.cc:1505] Processing CreateTablet for tablet 6b66d971101b4acdb207b7e7e36799d4 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:56.604447 22792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b66d971101b4acdb207b7e7e36799d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.603796 22791 tablet_service.cc:1505] Processing CreateTablet for tablet a67fb103d57c4633a7955f574084feb3 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:56.609555 22924 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.609613 22791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67fb103d57c4633a7955f574084feb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.609807 22793 tablet_service.cc:1505] Processing CreateTablet for tablet 75d3ebdb6f6e49829e45d892bb203c05 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:56.609930 22923 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.610162 22793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75d3ebdb6f6e49829e45d892bb203c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.611610 22924 tablet_bootstrap.cc:654] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.611933 22716 tablet_service.cc:1505] Processing CreateTablet for tablet a67fb103d57c4633a7955f574084feb3 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:56.612280 22716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67fb103d57c4633a7955f574084feb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.615262 22868 tablet_service.cc:1505] Processing CreateTablet for tablet cea9618bd841449b8db98622d5a12e2d (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:56.615414 22867 tablet_service.cc:1505] Processing CreateTablet for tablet 75d3ebdb6f6e49829e45d892bb203c05 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:56.615610 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea9618bd841449b8db98622d5a12e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.615634 22865 tablet_service.cc:1505] Processing CreateTablet for tablet a67fb103d57c4633a7955f574084feb3 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:56.615971 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67fb103d57c4633a7955f574084feb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.616134 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75d3ebdb6f6e49829e45d892bb203c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.615975 22866 tablet_service.cc:1505] Processing CreateTablet for tablet 6b66d971101b4acdb207b7e7e36799d4 (DEFAULT_TABLE table=test-workload [id=b9fd5345e94e4278924be996247e46cd]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:56.618516 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b66d971101b4acdb207b7e7e36799d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.611610 22923 tablet_bootstrap.cc:654] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.627588 22923 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.627702 22923 ts_tablet_manager.cc:1403] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20251023 02:10:56.627856 22924 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.627962 22924 ts_tablet_manager.cc:1403] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20251023 02:10:56.628052 22926 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.628463 22923 raft_consensus.cc:359] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.628623 22923 raft_consensus.cc:385] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.628644 22924 raft_consensus.cc:359] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.628717 22923 raft_consensus.cc:740] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.628768 22924 raft_consensus.cc:385] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.628844 22924 raft_consensus.cc:740] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.628911 22923 consensus_queue.cc:260] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.629007 22924 consensus_queue.cc:260] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.629369 22926 tablet_bootstrap.cc:654] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.629829 22924 ts_tablet_manager.cc:1434] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:56.630082 22923 ts_tablet_manager.cc:1434] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:56.630394 22924 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.630832 22926 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.630934 22926 ts_tablet_manager.cc:1403] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:56.631454 22923 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.631554 22926 raft_consensus.cc:359] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.631707 22926 raft_consensus.cc:385] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.631795 22926 raft_consensus.cc:740] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.631985 22926 consensus_queue.cc:260] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.633216 22923 tablet_bootstrap.cc:654] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.631588 22924 tablet_bootstrap.cc:654] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.633714 22926 ts_tablet_manager.cc:1434] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251023 02:10:56.634400 22926 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.635563 22923 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.635633 22926 tablet_bootstrap.cc:654] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.635673 22923 ts_tablet_manager.cc:1403] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:56.635723 22924 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.635830 22924 ts_tablet_manager.cc:1403] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20251023 02:10:56.636396 22923 raft_consensus.cc:359] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.636632 22923 raft_consensus.cc:385] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.636704 22923 raft_consensus.cc:740] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.636902 22923 consensus_queue.cc:260] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.636567 22924 raft_consensus.cc:359] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.637187 22924 raft_consensus.cc:385] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.637292 22924 raft_consensus.cc:740] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.637436 22923 ts_tablet_manager.cc:1434] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:56.637723 22923 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.637645 22924 consensus_queue.cc:260] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.638587 22924 ts_tablet_manager.cc:1434] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251023 02:10:56.638872 22924 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.639379 22923 tablet_bootstrap.cc:654] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.640301 22924 tablet_bootstrap.cc:654] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.645694 22923 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.645808 22923 ts_tablet_manager.cc:1403] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20251023 02:10:56.646024 22926 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.646154 22926 ts_tablet_manager.cc:1403] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.011s
I20251023 02:10:56.646584 22923 raft_consensus.cc:359] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.646708 22923 raft_consensus.cc:385] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.646756 22923 raft_consensus.cc:740] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.646925 22923 consensus_queue.cc:260] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.646947 22926 raft_consensus.cc:359] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.647138 22924 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.647091 22926 raft_consensus.cc:385] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.647305 22926 raft_consensus.cc:740] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.647351 22924 ts_tablet_manager.cc:1403] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20251023 02:10:56.647519 22926 consensus_queue.cc:260] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.647845 22923 ts_tablet_manager.cc:1434] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:10:56.647934 22924 raft_consensus.cc:359] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.648080 22923 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.648090 22924 raft_consensus.cc:385] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.648221 22924 raft_consensus.cc:740] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.648427 22924 consensus_queue.cc:260] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.649410 22923 tablet_bootstrap.cc:654] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.649482 22926 ts_tablet_manager.cc:1434] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251023 02:10:56.650031 22924 ts_tablet_manager.cc:1434] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251023 02:10:56.650331 22924 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.650837 22926 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.652319 22924 tablet_bootstrap.cc:654] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.652561 22926 tablet_bootstrap.cc:654] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.654377 22926 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.654475 22923 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.654506 22926 ts_tablet_manager.cc:1403] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 02:10:56.654636 22923 ts_tablet_manager.cc:1403] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.001s
I20251023 02:10:56.655001 22924 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.655105 22924 ts_tablet_manager.cc:1403] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251023 02:10:56.655035 22926 raft_consensus.cc:359] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.655189 22926 raft_consensus.cc:385] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.655257 22926 raft_consensus.cc:740] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.655248 22923 raft_consensus.cc:359] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.655444 22923 raft_consensus.cc:385] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.655503 22923 raft_consensus.cc:740] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.655480 22926 consensus_queue.cc:260] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.655687 22923 consensus_queue.cc:260] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.655781 22924 raft_consensus.cc:359] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.655907 22924 raft_consensus.cc:385] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.655966 22924 raft_consensus.cc:740] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.656126 22923 ts_tablet_manager.cc:1434] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:56.656153 22924 consensus_queue.cc:260] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.656546 22926 ts_tablet_manager.cc:1434] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 02:10:56.656785 22926 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.657413 22924 ts_tablet_manager.cc:1434] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 02:10:56.657946 22926 tablet_bootstrap.cc:654] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.659489 22926 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.659598 22926 ts_tablet_manager.cc:1403] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251023 02:10:56.660130 22926 raft_consensus.cc:359] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.660290 22926 raft_consensus.cc:385] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.660360 22926 raft_consensus.cc:740] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.660602 22926 consensus_queue.cc:260] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.661018 22926 ts_tablet_manager.cc:1434] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251023 02:10:56.677245 22928 raft_consensus.cc:493] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:56.677335 22928 raft_consensus.cc:515] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.677948 22928 leader_election.cc:290] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:56.685963 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a67fb103d57c4633a7955f574084feb3" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b" is_pre_election: true
I20251023 02:10:56.686102 22878 raft_consensus.cc:2468] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 0.
I20251023 02:10:56.686470 22769 leader_election.cc:304] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0; no voters:
I20251023 02:10:56.686477 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a67fb103d57c4633a7955f574084feb3" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992" is_pre_election: true
I20251023 02:10:56.686672 22729 raft_consensus.cc:2468] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 0.
I20251023 02:10:56.686722 22928 raft_consensus.cc:2804] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:56.686784 22928 raft_consensus.cc:493] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:56.686851 22928 raft_consensus.cc:3060] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.688671 22928 raft_consensus.cc:515] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.689078 22928 leader_election.cc:290] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:56.689347 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a67fb103d57c4633a7955f574084feb3" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992"
I20251023 02:10:56.689473 22729 raft_consensus.cc:3060] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.689486 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a67fb103d57c4633a7955f574084feb3" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b"
I20251023 02:10:56.689618 22878 raft_consensus.cc:3060] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.691927 22729 raft_consensus.cc:2468] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 1.
I20251023 02:10:56.691927 22878 raft_consensus.cc:2468] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 1.
I20251023 02:10:56.692258 22768 leader_election.cc:304] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:56.692443 22928 raft_consensus.cc:2804] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:56.692745 22928 raft_consensus.cc:697] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Running, Role: LEADER
I20251023 02:10:56.692955 22928 consensus_queue.cc:237] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.694926 22626 catalog_manager.cc:5649] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194). New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:56.773790 22931 raft_consensus.cc:493] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:56.773931 22931 raft_consensus.cc:515] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.774590 22931 leader_election.cc:290] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:56.780679 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b66d971101b4acdb207b7e7e36799d4" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992" is_pre_election: true
I20251023 02:10:56.780782 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b66d971101b4acdb207b7e7e36799d4" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0" is_pre_election: true
I20251023 02:10:56.780855 22729 raft_consensus.cc:2468] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 0.
I20251023 02:10:56.780984 22804 raft_consensus.cc:2468] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 0.
I20251023 02:10:56.781198 22842 leader_election.cc:304] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:56.781383 22931 raft_consensus.cc:2804] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:56.781453 22931 raft_consensus.cc:493] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:56.781512 22931 raft_consensus.cc:3060] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.783578 22931 raft_consensus.cc:515] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.784147 22931 leader_election.cc:290] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 election: Requested vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:56.784441 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b66d971101b4acdb207b7e7e36799d4" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992"
I20251023 02:10:56.784535 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b66d971101b4acdb207b7e7e36799d4" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0"
I20251023 02:10:56.784570 22729 raft_consensus.cc:3060] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.784667 22804 raft_consensus.cc:3060] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.786298 22729 raft_consensus.cc:2468] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 1.
I20251023 02:10:56.786440 22804 raft_consensus.cc:2468] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 1.
I20251023 02:10:56.786581 22842 leader_election.cc:304] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:56.786765 22931 raft_consensus.cc:2804] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:56.787056 22931 raft_consensus.cc:697] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Becoming Leader. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Running, Role: LEADER
I20251023 02:10:56.787292 22931 consensus_queue.cc:237] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.789278 22626 catalog_manager.cc:5649] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: term changed from 0 to 1, leader changed from <none> to 09b43b6f0a944b9096101927ba53a14b (127.15.161.195). New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:56.846743 22929 raft_consensus.cc:493] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:56.846868 22929 raft_consensus.cc:515] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.847602 22929 leader_election.cc:290] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:56.853104 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0" is_pre_election: true
I20251023 02:10:56.853116 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b" is_pre_election: true
I20251023 02:10:56.853272 22804 raft_consensus.cc:2468] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 0.
I20251023 02:10:56.853307 22878 raft_consensus.cc:2468] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 0.
I20251023 02:10:56.853634 22693 leader_election.cc:304] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:56.853847 22929 raft_consensus.cc:2804] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:56.853919 22929 raft_consensus.cc:493] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:56.854007 22929 raft_consensus.cc:3060] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.855698 22929 raft_consensus.cc:515] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.856155 22929 leader_election.cc:290] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 election: Requested vote from peers 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:56.856387 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b"
I20251023 02:10:56.856492 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0"
I20251023 02:10:56.856523 22878 raft_consensus.cc:3060] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.856626 22804 raft_consensus.cc:3060] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.858945 22804 raft_consensus.cc:2468] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1.
I20251023 02:10:56.858945 22878 raft_consensus.cc:2468] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1.
I20251023 02:10:56.859266 22693 leader_election.cc:304] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:56.859457 22929 raft_consensus.cc:2804] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:56.859756 22929 raft_consensus.cc:697] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Running, Role: LEADER
I20251023 02:10:56.859931 22929 consensus_queue.cc:237] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:56.861763 22626 catalog_manager.cc:5649] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fb388975cb842e7b476da64bd50a992 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "7fb388975cb842e7b476da64bd50a992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:56.870893 22928 raft_consensus.cc:493] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:56.871011 22928 raft_consensus.cc:515] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.871475 22928 leader_election.cc:290] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:56.871763 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75d3ebdb6f6e49829e45d892bb203c05" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992" is_pre_election: true
I20251023 02:10:56.871845 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75d3ebdb6f6e49829e45d892bb203c05" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b" is_pre_election: true
I20251023 02:10:56.871912 22729 raft_consensus.cc:2468] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 0.
I20251023 02:10:56.871945 22878 raft_consensus.cc:2468] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 0.
I20251023 02:10:56.872161 22769 leader_election.cc:304] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0; no voters:
I20251023 02:10:56.872336 22928 raft_consensus.cc:2804] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:56.872401 22928 raft_consensus.cc:493] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:56.872459 22928 raft_consensus.cc:3060] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.874133 22928 raft_consensus.cc:515] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.874588 22928 leader_election.cc:290] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:56.874897 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75d3ebdb6f6e49829e45d892bb203c05" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992"
I20251023 02:10:56.874953 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75d3ebdb6f6e49829e45d892bb203c05" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b"
I20251023 02:10:56.875000 22729 raft_consensus.cc:3060] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.875093 22878 raft_consensus.cc:3060] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:56.876585 22729 raft_consensus.cc:2468] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 1.
I20251023 02:10:56.876655 22878 raft_consensus.cc:2468] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 1.
I20251023 02:10:56.876871 22768 leader_election.cc:304] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:56.877095 22928 raft_consensus.cc:2804] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:56.877198 22928 raft_consensus.cc:697] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Running, Role: LEADER
I20251023 02:10:56.877360 22928 consensus_queue.cc:237] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.879245 22626 catalog_manager.cc:5649] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194). New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:56.886035 16007 test_util.cc:276] Using random seed: -242577139
I20251023 02:10:56.895376 22626 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53398:
name: "dugtrio"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 02:10:56.895937 22626 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table dugtrio in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 02:10:56.902144 22716 tablet_service.cc:1505] Processing CreateTablet for tablet 993893fc1d5c47b4a194340ad53ee8b9 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:56.902403 22793 tablet_service.cc:1505] Processing CreateTablet for tablet 993893fc1d5c47b4a194340ad53ee8b9 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:56.902588 22716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993893fc1d5c47b4a194340ad53ee8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.902732 22793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993893fc1d5c47b4a194340ad53ee8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.902660 22868 tablet_service.cc:1505] Processing CreateTablet for tablet 993893fc1d5c47b4a194340ad53ee8b9 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 02:10:56.903023 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993893fc1d5c47b4a194340ad53ee8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.907740 22926 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.907968 22923 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.908433 22867 tablet_service.cc:1505] Processing CreateTablet for tablet 7bed03c029ee46d1b69acf63e259e57b (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:56.908746 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bed03c029ee46d1b69acf63e259e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.908937 22792 tablet_service.cc:1505] Processing CreateTablet for tablet a20d5e5154ec47f59cf5acb61b3660f7 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:56.909093 22926 tablet_bootstrap.cc:654] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.909226 22792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a20d5e5154ec47f59cf5acb61b3660f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.909384 22923 tablet_bootstrap.cc:654] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.914486 22866 tablet_service.cc:1505] Processing CreateTablet for tablet a20d5e5154ec47f59cf5acb61b3660f7 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:56.914928 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a20d5e5154ec47f59cf5acb61b3660f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.915140 22718 tablet_service.cc:1505] Processing CreateTablet for tablet 7bed03c029ee46d1b69acf63e259e57b (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:56.915578 22718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bed03c029ee46d1b69acf63e259e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.918473 22794 tablet_service.cc:1505] Processing CreateTablet for tablet de0839aa60b24a9f8fdeefc2513d38d0 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:56.918689 22719 tablet_service.cc:1505] Processing CreateTablet for tablet a20d5e5154ec47f59cf5acb61b3660f7 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 02:10:56.918877 22794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0839aa60b24a9f8fdeefc2513d38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.919100 22719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a20d5e5154ec47f59cf5acb61b3660f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.920585 22923 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.920707 22923 ts_tablet_manager.cc:1403] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20251023 02:10:56.921303 22923 raft_consensus.cc:359] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.921445 22923 raft_consensus.cc:385] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.921514 22923 raft_consensus.cc:740] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.921737 22923 consensus_queue.cc:260] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.922508 22791 tablet_service.cc:1505] Processing CreateTablet for tablet 7bed03c029ee46d1b69acf63e259e57b (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 02:10:56.922722 22923 ts_tablet_manager.cc:1434] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 02:10:56.923020 22791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bed03c029ee46d1b69acf63e259e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.924579 22923 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.927970 22923 tablet_bootstrap.cc:654] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.930951 22924 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.932576 22924 tablet_bootstrap.cc:654] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.935591 22923 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.935777 22923 ts_tablet_manager.cc:1403] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20251023 02:10:56.936432 22923 raft_consensus.cc:359] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.936721 22923 raft_consensus.cc:385] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.936830 22923 raft_consensus.cc:740] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.937114 22923 consensus_queue.cc:260] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.937403 22868 tablet_service.cc:1505] Processing CreateTablet for tablet de0839aa60b24a9f8fdeefc2513d38d0 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:56.939853 22923 ts_tablet_manager.cc:1434] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251023 02:10:56.940012 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0839aa60b24a9f8fdeefc2513d38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.943071 22716 tablet_service.cc:1505] Processing CreateTablet for tablet de0839aa60b24a9f8fdeefc2513d38d0 (DEFAULT_TABLE table=dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 02:10:56.943531 22716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0839aa60b24a9f8fdeefc2513d38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:56.944859 22923 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.946513 22923 tablet_bootstrap.cc:654] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.947870 22924 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.948040 22924 ts_tablet_manager.cc:1403] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.002s
I20251023 02:10:56.948683 22924 raft_consensus.cc:359] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.948884 22924 raft_consensus.cc:385] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.949016 22924 raft_consensus.cc:740] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.949235 22924 consensus_queue.cc:260] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.949844 22924 ts_tablet_manager.cc:1434] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 02:10:56.949874 22926 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.950098 22926 ts_tablet_manager.cc:1403] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.042s user 0.024s sys 0.018s
I20251023 02:10:56.950194 22924 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.951030 22926 raft_consensus.cc:359] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.951310 22926 raft_consensus.cc:385] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.951460 22926 raft_consensus.cc:740] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.951565 22924 tablet_bootstrap.cc:654] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.951781 22926 consensus_queue.cc:260] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.955031 22926 ts_tablet_manager.cc:1434] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251023 02:10:56.955758 22926 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.956462 22924 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.956609 22924 ts_tablet_manager.cc:1403] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20251023 02:10:56.957059 22926 tablet_bootstrap.cc:654] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.957170 22924 raft_consensus.cc:359] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.957336 22924 raft_consensus.cc:385] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.957410 22924 raft_consensus.cc:740] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.957479 22923 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.957626 22923 ts_tablet_manager.cc:1403] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.003s
I20251023 02:10:56.957677 22924 consensus_queue.cc:260] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.958299 22924 ts_tablet_manager.cc:1434] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:56.958449 22923 raft_consensus.cc:359] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.958597 22924 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.958652 22923 raft_consensus.cc:385] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.958801 22923 raft_consensus.cc:740] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.959065 22926 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.959064 22923 consensus_queue.cc:260] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.959280 22926 ts_tablet_manager.cc:1403] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:56.959940 22924 tablet_bootstrap.cc:654] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.959918 22926 raft_consensus.cc:359] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.960120 22926 raft_consensus.cc:385] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.960222 22926 raft_consensus.cc:740] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.960515 22926 consensus_queue.cc:260] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.961508 22923 ts_tablet_manager.cc:1434] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251023 02:10:56.962139 22926 ts_tablet_manager.cc:1434] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251023 02:10:56.963081 22926 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.964605 22926 tablet_bootstrap.cc:654] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.964622 22923 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:56.966270 22923 tablet_bootstrap.cc:654] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.967986 22924 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.968127 22924 ts_tablet_manager.cc:1403] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20251023 02:10:56.968385 22923 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: No bootstrap required, opened a new log
I20251023 02:10:56.968453 22926 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.968554 22926 ts_tablet_manager.cc:1403] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251023 02:10:56.968565 22923 ts_tablet_manager.cc:1403] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20251023 02:10:56.968801 22924 raft_consensus.cc:359] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.968983 22924 raft_consensus.cc:385] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.969072 22924 raft_consensus.cc:740] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.969215 22926 raft_consensus.cc:359] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.969274 22924 consensus_queue.cc:260] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.969331 22923 raft_consensus.cc:359] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.969404 22926 raft_consensus.cc:385] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.969528 22923 raft_consensus.cc:385] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.969556 22926 raft_consensus.cc:740] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.969609 22923 raft_consensus.cc:740] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.969830 22924 ts_tablet_manager.cc:1434] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 02:10:56.969983 22926 consensus_queue.cc:260] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.970172 22924 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Bootstrap starting.
I20251023 02:10:56.970557 22926 ts_tablet_manager.cc:1434] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:56.970829 22926 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Bootstrap starting.
I20251023 02:10:56.970178 22923 consensus_queue.cc:260] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.971407 22923 ts_tablet_manager.cc:1434] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251023 02:10:56.971429 22924 tablet_bootstrap.cc:654] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.972285 22926 tablet_bootstrap.cc:654] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Neither blocks nor log segments found. Creating new log.
I20251023 02:10:56.974892 22926 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: No bootstrap required, opened a new log
I20251023 02:10:56.975031 22926 ts_tablet_manager.cc:1403] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20251023 02:10:56.975608 22926 raft_consensus.cc:359] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.975797 22926 raft_consensus.cc:385] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.975876 22926 raft_consensus.cc:740] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.976095 22926 consensus_queue.cc:260] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.976629 22926 ts_tablet_manager.cc:1434] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 02:10:56.976686 22924 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: No bootstrap required, opened a new log
I20251023 02:10:56.976819 22924 ts_tablet_manager.cc:1403] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20251023 02:10:56.977257 22924 raft_consensus.cc:359] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.977391 22924 raft_consensus.cc:385] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 02:10:56.977458 22924 raft_consensus.cc:740] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Initialized, Role: FOLLOWER
I20251023 02:10:56.977653 22924 consensus_queue.cc:260] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:56.978018 22924 ts_tablet_manager.cc:1434] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 02:10:57.005513 22929 raft_consensus.cc:493] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:57.005662 22929 raft_consensus.cc:515] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.006090 22929 leader_election.cc:290] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:57.006464 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7bed03c029ee46d1b69acf63e259e57b" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b" is_pre_election: true
I20251023 02:10:57.006609 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7bed03c029ee46d1b69acf63e259e57b" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0" is_pre_election: true
I20251023 02:10:57.006659 22878 raft_consensus.cc:2468] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 0.
I20251023 02:10:57.006846 22804 raft_consensus.cc:2468] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 0.
I20251023 02:10:57.007102 22694 leader_election.cc:304] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:57.007428 22929 raft_consensus.cc:2804] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:57.007540 22929 raft_consensus.cc:493] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:57.007659 22929 raft_consensus.cc:3060] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.009349 22929 raft_consensus.cc:515] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.009861 22929 leader_election.cc:290] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 election: Requested vote from peers 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:57.010309 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7bed03c029ee46d1b69acf63e259e57b" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b"
I20251023 02:10:57.010329 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7bed03c029ee46d1b69acf63e259e57b" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0"
I20251023 02:10:57.010476 22878 raft_consensus.cc:3060] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.010512 22804 raft_consensus.cc:3060] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.012264 22878 raft_consensus.cc:2468] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1.
I20251023 02:10:57.012285 22804 raft_consensus.cc:2468] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1.
I20251023 02:10:57.012624 22693 leader_election.cc:304] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:57.012964 22929 raft_consensus.cc:2804] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:57.013146 22929 raft_consensus.cc:697] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Running, Role: LEADER
I20251023 02:10:57.013490 22929 consensus_queue.cc:237] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.015681 22626 catalog_manager.cc:5649] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fb388975cb842e7b476da64bd50a992 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "7fb388975cb842e7b476da64bd50a992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:57.092866 22931 raft_consensus.cc:493] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:57.093056 22931 raft_consensus.cc:515] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.093546 22931 leader_election.cc:290] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:57.093961 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a20d5e5154ec47f59cf5acb61b3660f7" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992" is_pre_election: true
I20251023 02:10:57.094049 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a20d5e5154ec47f59cf5acb61b3660f7" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0" is_pre_election: true
I20251023 02:10:57.094172 22729 raft_consensus.cc:2468] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 0.
I20251023 02:10:57.094296 22804 raft_consensus.cc:2468] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 0.
I20251023 02:10:57.094560 22842 leader_election.cc:304] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:57.094802 22931 raft_consensus.cc:2804] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:57.094894 22931 raft_consensus.cc:493] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:57.094995 22931 raft_consensus.cc:3060] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.096092 22928 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:57.097150 22931 raft_consensus.cc:515] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.097759 22931 leader_election.cc:290] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 election: Requested vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:57.098158 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a20d5e5154ec47f59cf5acb61b3660f7" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0"
I20251023 02:10:57.098320 22804 raft_consensus.cc:3060] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.101014 22804 raft_consensus.cc:2468] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 1.
I20251023 02:10:57.101346 22842 leader_election.cc:304] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0; no voters:
I20251023 02:10:57.101583 22931 raft_consensus.cc:2804] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:57.101578 22729 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a20d5e5154ec47f59cf5acb61b3660f7" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992"
I20251023 02:10:57.101799 22931 raft_consensus.cc:697] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Becoming Leader. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Running, Role: LEADER
I20251023 02:10:57.101981 22729 raft_consensus.cc:3060] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.102070 22931 consensus_queue.cc:237] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.101989 22929 raft_consensus.cc:493] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:57.102380 22928 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:57.102504 22929 raft_consensus.cc:515] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.103829 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0839aa60b24a9f8fdeefc2513d38d0" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0" is_pre_election: true
I20251023 02:10:57.104136 22804 raft_consensus.cc:2468] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 0.
I20251023 02:10:57.104473 22693 leader_election.cc:304] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:57.104650 22729 raft_consensus.cc:2468] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 1.
I20251023 02:10:57.104764 22951 raft_consensus.cc:2804] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:57.104858 22951 raft_consensus.cc:493] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:57.104936 22951 raft_consensus.cc:3060] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.106588 22951 raft_consensus.cc:515] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.107007 22951 leader_election.cc:290] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 election: Requested vote from peers 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:57.107416 22878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0839aa60b24a9f8fdeefc2513d38d0" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b"
I20251023 02:10:57.107584 22878 raft_consensus.cc:3060] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.107764 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0839aa60b24a9f8fdeefc2513d38d0" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0"
I20251023 02:10:57.107908 22804 raft_consensus.cc:3060] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.110216 22878 raft_consensus.cc:2468] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1.
I20251023 02:10:57.110509 22694 leader_election.cc:304] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:57.110747 22951 raft_consensus.cc:2804] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:57.110862 22877 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0839aa60b24a9f8fdeefc2513d38d0" candidate_uuid: "7fb388975cb842e7b476da64bd50a992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b" is_pre_election: true
I20251023 02:10:57.111028 22877 raft_consensus.cc:2376] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1. Re-sending same reply.
I20251023 02:10:57.111517 22929 leader_election.cc:290] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:57.112185 22804 raft_consensus.cc:2468] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb388975cb842e7b476da64bd50a992 in term 1.
I20251023 02:10:57.113008 22951 raft_consensus.cc:697] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Running, Role: LEADER
I20251023 02:10:57.113256 22951 consensus_queue.cc:237] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.114606 22626 catalog_manager.cc:5649] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: term changed from 0 to 1, leader changed from <none> to 09b43b6f0a944b9096101927ba53a14b (127.15.161.195). New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:57.119868 22626 catalog_manager.cc:5649] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fb388975cb842e7b476da64bd50a992 (127.15.161.193). New cstate: current_term: 1 leader_uuid: "7fb388975cb842e7b476da64bd50a992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:57.121827 22928 raft_consensus.cc:493] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:57.121968 22928 raft_consensus.cc:515] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.122460 22928 leader_election.cc:290] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:57.122766 22728 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993893fc1d5c47b4a194340ad53ee8b9" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992" is_pre_election: true
I20251023 02:10:57.122844 22877 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993893fc1d5c47b4a194340ad53ee8b9" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b" is_pre_election: true
I20251023 02:10:57.122936 22728 raft_consensus.cc:2468] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 0.
I20251023 02:10:57.123029 22877 raft_consensus.cc:2468] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 0.
I20251023 02:10:57.123302 22768 leader_election.cc:304] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e35508ce4004db383775ecd53a92ba0, 7fb388975cb842e7b476da64bd50a992; no voters:
I20251023 02:10:57.123507 22928 raft_consensus.cc:2804] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 02:10:57.123601 22928 raft_consensus.cc:493] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:57.123663 22928 raft_consensus.cc:3060] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.125216 22928 raft_consensus.cc:515] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.125623 22928 leader_election.cc:290] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:57.125936 22728 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993893fc1d5c47b4a194340ad53ee8b9" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992"
I20251023 02:10:57.125998 22877 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "993893fc1d5c47b4a194340ad53ee8b9" candidate_uuid: "6e35508ce4004db383775ecd53a92ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b43b6f0a944b9096101927ba53a14b"
I20251023 02:10:57.126070 22728 raft_consensus.cc:3060] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.126154 22877 raft_consensus.cc:3060] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 0 FOLLOWER]: Advancing to term 1
I20251023 02:10:57.127969 22877 raft_consensus.cc:2468] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 1.
I20251023 02:10:57.128067 22728 raft_consensus.cc:2468] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e35508ce4004db383775ecd53a92ba0 in term 1.
I20251023 02:10:57.128265 22769 leader_election.cc:304] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0; no voters:
I20251023 02:10:57.128492 22928 raft_consensus.cc:2804] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 02:10:57.128626 22928 raft_consensus.cc:697] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e35508ce4004db383775ecd53a92ba0, State: Running, Role: LEADER
I20251023 02:10:57.128829 22928 consensus_queue.cc:237] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:57.130609 22626 catalog_manager.cc:5649] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194). New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:57.162544 16007 tablet_server.cc:178] TabletServer@127.15.161.193:0 shutting down...
I20251023 02:10:57.170300 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:10:57.170571 16007 tablet_replica.cc:333] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.177148 16007 raft_consensus.cc:2243] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:57.177325 16007 pending_rounds.cc:70] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Trying to abort 1 pending ops.
I20251023 02:10:57.177358 16007 pending_rounds.cc:77] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815632338034688 op_type: NO_OP noop_request { }
I20251023 02:10:57.177421 16007 raft_consensus.cc:2889] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:57.177474 16007 raft_consensus.cc:2272] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.178158 16007 tablet_replica.cc:333] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.178282 16007 raft_consensus.cc:2243] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:57.178372 16007 raft_consensus.cc:2272] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.178786 16007 tablet_replica.cc:333] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.178890 16007 raft_consensus.cc:2243] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:57.178972 16007 raft_consensus.cc:2272] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.179276 16007 tablet_replica.cc:333] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.179378 16007 raft_consensus.cc:2243] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:57.179450 16007 raft_consensus.cc:2272] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.179803 16007 tablet_replica.cc:333] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.179889 16007 raft_consensus.cc:2243] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:57.180002 16007 pending_rounds.cc:70] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Trying to abort 1 pending ops.
I20251023 02:10:57.180030 16007 pending_rounds.cc:77] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815631300132864 op_type: NO_OP noop_request { }
I20251023 02:10:57.180081 16007 raft_consensus.cc:2889] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:57.180126 16007 raft_consensus.cc:2272] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.180533 16007 tablet_replica.cc:333] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.180613 16007 raft_consensus.cc:2243] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:57.180701 16007 raft_consensus.cc:2272] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.181093 16007 tablet_replica.cc:333] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.181172 16007 raft_consensus.cc:2243] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:57.181246 16007 raft_consensus.cc:2272] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.181533 16007 tablet_replica.cc:333] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: stopping tablet replica
I20251023 02:10:57.181608 16007 raft_consensus.cc:2243] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:57.181738 16007 pending_rounds.cc:70] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Trying to abort 1 pending ops.
I20251023 02:10:57.181766 16007 pending_rounds.cc:77] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815631929606144 op_type: NO_OP noop_request { }
I20251023 02:10:57.181819 16007 raft_consensus.cc:2889] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:57.181865 16007 raft_consensus.cc:2272] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:57.186903 16007 tablet_server.cc:195] TabletServer@127.15.161.193:0 shutdown complete.
W20251023 02:10:57.227921 22842 consensus_peers.cc:597] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b -> Peer 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): Couldn't send request to peer 7fb388975cb842e7b476da64bd50a992. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 02:10:57.258514 22768 consensus_peers.cc:597] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 -> Peer 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): Couldn't send request to peer 7fb388975cb842e7b476da64bd50a992. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:57.397158 22931 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:57.457043 22928 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 02:10:57.492925 22842 consensus_peers.cc:597] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b -> Peer 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): Couldn't send request to peer 7fb388975cb842e7b476da64bd50a992. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 02:10:57.546720 22768 consensus_peers.cc:597] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 -> Peer 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): Couldn't send request to peer 7fb388975cb842e7b476da64bd50a992. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 02:10:57.585403 22768 consensus_peers.cc:597] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 -> Peer 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): Couldn't send request to peer 7fb388975cb842e7b476da64bd50a992. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:57.585497 22947 consensus_queue.cc:1048] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:57.662454 22943 consensus_queue.cc:1048] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:57.826000 22931 consensus_queue.cc:579] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Leader has been unable to successfully communicate with peer 7fb388975cb842e7b476da64bd50a992 for more than 1 seconds (1.038s)
I20251023 02:10:58.197012 22627 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53396:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251023 02:10:58.197170 22627 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53396:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251023 02:10:58.201197 22627 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Sending DeleteTablet for 3 replicas of tablet 993893fc1d5c47b4a194340ad53ee8b9
I20251023 02:10:58.201668 22627 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Sending DeleteTablet for 3 replicas of tablet de0839aa60b24a9f8fdeefc2513d38d0
I20251023 02:10:58.202045 22868 tablet_service.cc:1552] Processing DeleteTablet for tablet 993893fc1d5c47b4a194340ad53ee8b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:54602
I20251023 02:10:58.202167 22627 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Sending DeleteTablet for 3 replicas of tablet 7bed03c029ee46d1b69acf63e259e57b
I20251023 02:10:58.202564 22627 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: Sending DeleteTablet for 3 replicas of tablet a20d5e5154ec47f59cf5acb61b3660f7
I20251023 02:10:58.203622 22867 tablet_service.cc:1552] Processing DeleteTablet for tablet de0839aa60b24a9f8fdeefc2513d38d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:54602
I20251023 02:10:58.203832 22997 tablet_replica.cc:333] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:10:58.203845 22866 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bed03c029ee46d1b69acf63e259e57b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:54602
I20251023 02:10:58.203922 22867 tablet_service.cc:1552] Processing DeleteTablet for tablet a20d5e5154ec47f59cf5acb61b3660f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:54602
I20251023 02:10:58.204116 22997 raft_consensus.cc:2243] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.204290 22997 raft_consensus.cc:2272] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.204623 22792 tablet_service.cc:1552] Processing DeleteTablet for tablet 993893fc1d5c47b4a194340ad53ee8b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:46556
I20251023 02:10:58.204825 22791 tablet_service.cc:1552] Processing DeleteTablet for tablet de0839aa60b24a9f8fdeefc2513d38d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:46556
I20251023 02:10:58.205062 22794 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bed03c029ee46d1b69acf63e259e57b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:46556
I20251023 02:10:58.205276 22791 tablet_service.cc:1552] Processing DeleteTablet for tablet a20d5e5154ec47f59cf5acb61b3660f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:46556
I20251023 02:10:58.205780 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 02:10:58.205888 22998 tablet_replica.cc:333] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:10:58.206027 22998 raft_consensus.cc:2243] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:58.206285 22997 ts_tablet_manager.cc:1916] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.206334 22998 raft_consensus.cc:2272] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251023 02:10:58.206837 22612 catalog_manager.cc:4712] TS 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 993893fc1d5c47b4a194340ad53ee8b9: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111)
I20251023 02:10:58.208636 22998 ts_tablet_manager.cc:1916] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251023 02:10:58.209208 23001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:58.210987 23000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:58.211985 22997 ts_tablet_manager.cc:1929] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 02:10:58.212145 22997 log.cc:1199] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/993893fc1d5c47b4a194340ad53ee8b9
I20251023 02:10:58.212694 22997 ts_tablet_manager.cc:1950] T 993893fc1d5c47b4a194340ad53ee8b9 P 09b43b6f0a944b9096101927ba53a14b: Deleting consensus metadata
I20251023 02:10:58.213973 22997 tablet_replica.cc:333] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:10:58.214015 22613 catalog_manager.cc:4985] TS 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867): tablet 993893fc1d5c47b4a194340ad53ee8b9 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.214227 22997 raft_consensus.cc:2243] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.214380 22997 raft_consensus.cc:2272] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.215054 22997 ts_tablet_manager.cc:1916] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.217096 22998 ts_tablet_manager.cc:1929] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 02:10:58.217224 22998 log.cc:1199] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/993893fc1d5c47b4a194340ad53ee8b9
I20251023 02:10:58.217658 22998 ts_tablet_manager.cc:1950] T 993893fc1d5c47b4a194340ad53ee8b9 P 6e35508ce4004db383775ecd53a92ba0: Deleting consensus metadata
I20251023 02:10:58.218866 22612 catalog_manager.cc:4985] TS 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611): tablet 993893fc1d5c47b4a194340ad53ee8b9 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.219161 22998 tablet_replica.cc:333] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:10:58.219341 22998 raft_consensus.cc:2243] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.219477 22998 raft_consensus.cc:2272] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.220059 22998 ts_tablet_manager.cc:1916] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.221944 22997 ts_tablet_manager.cc:1929] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 02:10:58.222085 22997 log.cc:1199] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/de0839aa60b24a9f8fdeefc2513d38d0
I20251023 02:10:58.222651 22997 ts_tablet_manager.cc:1950] T de0839aa60b24a9f8fdeefc2513d38d0 P 09b43b6f0a944b9096101927ba53a14b: Deleting consensus metadata
I20251023 02:10:58.223909 22613 catalog_manager.cc:4985] TS 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867): tablet de0839aa60b24a9f8fdeefc2513d38d0 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.224291 22997 tablet_replica.cc:333] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:10:58.224503 22997 raft_consensus.cc:2243] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.224696 22997 raft_consensus.cc:2272] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.225389 22997 ts_tablet_manager.cc:1916] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251023 02:10:58.227182 23003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:58.227344 22998 ts_tablet_manager.cc:1929] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 02:10:58.227449 22998 log.cc:1199] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/de0839aa60b24a9f8fdeefc2513d38d0
I20251023 02:10:58.227296 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:58.227744 22998 ts_tablet_manager.cc:1950] T de0839aa60b24a9f8fdeefc2513d38d0 P 6e35508ce4004db383775ecd53a92ba0: Deleting consensus metadata
I20251023 02:10:58.227777 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:58.227855 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:58.227900 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185458227896 us; error 0 us; skew 500 ppm
I20251023 02:10:58.228049 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:58.228464 22998 tablet_replica.cc:333] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:10:58.228525 22612 catalog_manager.cc:4985] TS 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611): tablet de0839aa60b24a9f8fdeefc2513d38d0 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.228619 22998 raft_consensus.cc:2243] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.228744 22998 raft_consensus.cc:2272] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.229204 22998 ts_tablet_manager.cc:1916] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.229594 16007 webserver.cc:492] Webserver started at http://127.15.161.196:43741/ using document root <none> and password file <none>
I20251023 02:10:58.229838 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:58.229920 16007 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 02:10:58.230054 16007 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 02:10:58.230753 16007 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-3-root/instance:
uuid: "c3864b87aaa34f92b748ea4eff18d150"
format_stamp: "Formatted at 2025-10-23 02:10:58 on dist-test-slave-bfbt"
I20251023 02:10:58.231701 22997 ts_tablet_manager.cc:1929] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 02:10:58.231789 22997 log.cc:1199] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/7bed03c029ee46d1b69acf63e259e57b
I20251023 02:10:58.232041 22997 ts_tablet_manager.cc:1950] T 7bed03c029ee46d1b69acf63e259e57b P 09b43b6f0a944b9096101927ba53a14b: Deleting consensus metadata
I20251023 02:10:58.232942 22613 catalog_manager.cc:4985] TS 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867): tablet 7bed03c029ee46d1b69acf63e259e57b (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.232990 22997 tablet_replica.cc:333] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:10:58.233142 22997 raft_consensus.cc:2243] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:10:58.233472 22997 raft_consensus.cc:2272] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.233521 22998 ts_tablet_manager.cc:1929] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 02:10:58.233613 22998 log.cc:1199] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/7bed03c029ee46d1b69acf63e259e57b
I20251023 02:10:58.233892 22998 ts_tablet_manager.cc:1950] T 7bed03c029ee46d1b69acf63e259e57b P 6e35508ce4004db383775ecd53a92ba0: Deleting consensus metadata
I20251023 02:10:58.234167 16007 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251023 02:10:58.234241 22997 ts_tablet_manager.cc:1916] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.234643 22612 catalog_manager.cc:4985] TS 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611): tablet 7bed03c029ee46d1b69acf63e259e57b (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.234730 22998 tablet_replica.cc:333] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:10:58.234889 22998 raft_consensus.cc:2243] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.235034 22998 raft_consensus.cc:2272] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.235616 22998 ts_tablet_manager.cc:1916] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.237674 23008 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:58.238032 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:58.238149 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-3-root
uuid: "c3864b87aaa34f92b748ea4eff18d150"
format_stamp: "Formatted at 2025-10-23 02:10:58 on dist-test-slave-bfbt"
I20251023 02:10:58.238247 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:58.240141 22998 ts_tablet_manager.cc:1929] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 02:10:58.240151 22997 ts_tablet_manager.cc:1929] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 02:10:58.240242 22998 log.cc:1199] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/a20d5e5154ec47f59cf5acb61b3660f7
I20251023 02:10:58.240335 22997 log.cc:1199] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/a20d5e5154ec47f59cf5acb61b3660f7
I20251023 02:10:58.240584 22998 ts_tablet_manager.cc:1950] T a20d5e5154ec47f59cf5acb61b3660f7 P 6e35508ce4004db383775ecd53a92ba0: Deleting consensus metadata
I20251023 02:10:58.240720 22997 ts_tablet_manager.cc:1950] T a20d5e5154ec47f59cf5acb61b3660f7 P 09b43b6f0a944b9096101927ba53a14b: Deleting consensus metadata
I20251023 02:10:58.241328 22612 catalog_manager.cc:4985] TS 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611): tablet a20d5e5154ec47f59cf5acb61b3660f7 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.241663 22613 catalog_manager.cc:4985] TS 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867): tablet a20d5e5154ec47f59cf5acb61b3660f7 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.258085 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:58.258442 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:58.258843 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:58.259511 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 02:10:58.259562 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:58.259630 16007 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 02:10:58.259680 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:58.269979 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.196:36489
I20251023 02:10:58.269996 23070 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.196:36489 every 8 connection(s)
I20251023 02:10:58.271333 16007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 02:10:58.273989 23074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:58.274642 23075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 02:10:58.277254 23077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 02:10:58.277604 16007 server_base.cc:1047] running on GCE node
I20251023 02:10:58.277712 23071 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34399
I20251023 02:10:58.277830 23071 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:58.277850 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 02:10:58.277899 16007 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 02:10:58.277933 16007 hybrid_clock.cc:648] HybridClock initialized: now 1761185458277929 us; error 0 us; skew 500 ppm
I20251023 02:10:58.278062 16007 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 02:10:58.278149 23071 heartbeater.cc:507] Master 127.15.161.254:34399 requested a full tablet report, sending...
I20251023 02:10:58.279038 22627 ts_manager.cc:194] Registered new tserver with Master: c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489)
I20251023 02:10:58.279342 16007 webserver.cc:492] Webserver started at http://127.15.161.193:39345/ using document root <none> and password file <none>
I20251023 02:10:58.279554 16007 fs_manager.cc:362] Metadata directory not provided
I20251023 02:10:58.279620 16007 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 02:10:58.279996 22627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53414
I20251023 02:10:58.281536 16007 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:58.283078 23082 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 02:10:58.283425 16007 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:58.283526 16007 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root
uuid: "7fb388975cb842e7b476da64bd50a992"
format_stamp: "Formatted at 2025-10-23 02:10:55 on dist-test-slave-bfbt"
I20251023 02:10:58.283629 16007 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 02:10:58.297482 16007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 02:10:58.297866 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 02:10:58.298313 16007 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 02:10:58.299259 23089 ts_tablet_manager.cc:548] Loading tablet metadata (0/8 complete)
I20251023 02:10:58.314586 16007 ts_tablet_manager.cc:585] Loaded tablet metadata (8 total tablets, 8 live tablets)
I20251023 02:10:58.314659 16007 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.016s user 0.000s sys 0.001s
I20251023 02:10:58.314718 16007 ts_tablet_manager.cc:600] Registering tablets (0/8 complete)
I20251023 02:10:58.316534 23089 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.319520 23089 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:58.319772 23089 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.319950 23089 ts_tablet_manager.cc:1403] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 02:10:58.320380 23089 raft_consensus.cc:359] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.320511 23089 raft_consensus.cc:740] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.320711 23089 consensus_queue.cc:260] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.321312 23089 ts_tablet_manager.cc:1434] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:58.321496 23089 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.324519 23089 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:58.324815 23089 tablet_bootstrap.cc:492] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.324994 23089 ts_tablet_manager.cc:1403] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:58.325477 23089 raft_consensus.cc:359] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:58.325634 23089 raft_consensus.cc:740] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.325803 23089 consensus_queue.cc:260] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:58.326157 23089 ts_tablet_manager.cc:1434] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:58.326341 23089 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.327323 16007 ts_tablet_manager.cc:616] Registered 8 tablets
I20251023 02:10:58.327391 16007 ts_tablet_manager.cc:595] Time spent register tablets: real 0.013s user 0.011s sys 0.000s
I20251023 02:10:58.329521 23089 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:58.329835 23089 tablet_bootstrap.cc:492] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.330040 23089 ts_tablet_manager.cc:1403] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 02:10:58.330710 23089 raft_consensus.cc:359] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.330900 23089 raft_consensus.cc:740] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.331166 23089 consensus_queue.cc:260] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.331570 23089 ts_tablet_manager.cc:1434] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 02:10:58.331795 23089 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.336386 23089 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 02:10:58.336664 22928 consensus_queue.cc:579] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Leader has been unable to successfully communicate with peer 7fb388975cb842e7b476da64bd50a992 for more than 1 seconds (1.459s)
I20251023 02:10:58.336709 23089 tablet_bootstrap.cc:492] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.336905 23089 ts_tablet_manager.cc:1403] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251023 02:10:58.337406 23089 raft_consensus.cc:359] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.337579 23089 raft_consensus.cc:740] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.337754 23089 consensus_queue.cc:260] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.338083 23089 ts_tablet_manager.cc:1434] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:58.338369 23089 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.342417 22804 consensus_queue.cc:237] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:58.342996 23089 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:58.343287 23089 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.343483 23089 ts_tablet_manager.cc:1403] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251023 02:10:58.343998 23089 raft_consensus.cc:359] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.344164 23089 raft_consensus.cc:740] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.344364 23089 consensus_queue.cc:260] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.344803 23089 ts_tablet_manager.cc:1434] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251023 02:10:58.344991 23089 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.345278 22877 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:58.345893 22987 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:58.348225 22987 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.353035 22877 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.354633 23089 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:58.354952 23089 tablet_bootstrap.cc:492] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.355131 23089 ts_tablet_manager.cc:1403] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20251023 02:10:58.355636 23089 raft_consensus.cc:359] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.355751 23089 raft_consensus.cc:740] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.355949 23089 consensus_queue.cc:260] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.356366 23089 ts_tablet_manager.cc:1434] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:58.356572 23089 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.357384 22612 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 75d3ebdb6f6e49829e45d892bb203c05 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 02:10:58.359355 22627 catalog_manager.cc:5649] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.363559 23089 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:58.363915 23089 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.364110 23089 ts_tablet_manager.cc:1403] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
W20251023 02:10:58.364261 22675 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 7fb388975cb842e7b476da64bd50a992 not available for placement
I20251023 02:10:58.364738 23089 raft_consensus.cc:359] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.364864 23089 raft_consensus.cc:740] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.365060 23089 consensus_queue.cc:260] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.365644 23089 ts_tablet_manager.cc:1434] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:58.365825 23089 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Bootstrap starting.
I20251023 02:10:58.369208 23089 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:58.369451 23089 tablet_bootstrap.cc:492] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Bootstrap complete.
I20251023 02:10:58.369614 23089 ts_tablet_manager.cc:1403] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 02:10:58.370021 23089 raft_consensus.cc:359] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.370158 23089 raft_consensus.cc:740] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb388975cb842e7b476da64bd50a992, State: Initialized, Role: FOLLOWER
I20251023 02:10:58.370340 23089 consensus_queue.cc:260] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.370703 23089 ts_tablet_manager.cc:1434] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
W20251023 02:10:58.374881 22768 consensus_peers.cc:597] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 -> Peer 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): Couldn't send request to peer 7fb388975cb842e7b476da64bd50a992. Status: Network error: Client connection negotiation failed: client connection to 127.15.161.193:38447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 02:10:58.377688 22768 consensus_peers.cc:597] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 -> Peer c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): Couldn't send request to peer c3864b87aaa34f92b748ea4eff18d150. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75d3ebdb6f6e49829e45d892bb203c05. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:58.378866 16007 rpc_server.cc:307] RPC server started. Bound to: 127.15.161.193:38447
I20251023 02:10:58.378907 23156 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.161.193:38447 every 8 connection(s)
I20251023 02:10:58.384361 23157 heartbeater.cc:344] Connected to a master server at 127.15.161.254:34399
I20251023 02:10:58.384460 23157 heartbeater.cc:461] Registering TS with master...
I20251023 02:10:58.384734 23157 heartbeater.cc:507] Master 127.15.161.254:34399 requested a full tablet report, sending...
I20251023 02:10:58.385782 22627 ts_manager.cc:194] Re-registered known tserver with Master: 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447)
I20251023 02:10:58.388090 22627 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53430
I20251023 02:10:58.403203 23115 tablet_service.cc:1552] Processing DeleteTablet for tablet a20d5e5154ec47f59cf5acb61b3660f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:60714
I20251023 02:10:58.403241 23114 tablet_service.cc:1552] Processing DeleteTablet for tablet de0839aa60b24a9f8fdeefc2513d38d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:60714
I20251023 02:10:58.403684 23161 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:58.403815 23161 raft_consensus.cc:2243] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.403964 23161 raft_consensus.cc:2272] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.404560 23161 ts_tablet_manager.cc:1916] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.404976 23115 tablet_service.cc:1552] Processing DeleteTablet for tablet 993893fc1d5c47b4a194340ad53ee8b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:60714
I20251023 02:10:58.408512 23161 ts_tablet_manager.cc:1929] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 02:10:58.408596 23161 log.cc:1199] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/a20d5e5154ec47f59cf5acb61b3660f7
I20251023 02:10:58.408834 23161 ts_tablet_manager.cc:1950] T a20d5e5154ec47f59cf5acb61b3660f7 P 7fb388975cb842e7b476da64bd50a992: Deleting consensus metadata
I20251023 02:10:58.409466 23161 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:58.409566 22612 catalog_manager.cc:4985] TS 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): tablet a20d5e5154ec47f59cf5acb61b3660f7 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.409644 23161 raft_consensus.cc:2243] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.409771 23161 pending_rounds.cc:70] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Trying to abort 1 pending ops.
I20251023 02:10:58.409842 23161 pending_rounds.cc:77] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815632338034688 op_type: NO_OP noop_request { }
I20251023 02:10:58.409940 23161 raft_consensus.cc:2889] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:58.410045 23161 raft_consensus.cc:2272] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.410567 23161 ts_tablet_manager.cc:1916] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.424594 23161 ts_tablet_manager.cc:1929] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 02:10:58.424670 23161 log.cc:1199] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/de0839aa60b24a9f8fdeefc2513d38d0
I20251023 02:10:58.424902 23161 ts_tablet_manager.cc:1950] T de0839aa60b24a9f8fdeefc2513d38d0 P 7fb388975cb842e7b476da64bd50a992: Deleting consensus metadata
I20251023 02:10:58.425472 23161 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:58.425568 22612 catalog_manager.cc:4985] TS 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): tablet de0839aa60b24a9f8fdeefc2513d38d0 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.425609 23161 raft_consensus.cc:2243] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.425711 23161 raft_consensus.cc:2272] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.426213 23161 ts_tablet_manager.cc:1916] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.429440 23161 ts_tablet_manager.cc:1929] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 02:10:58.429510 23161 log.cc:1199] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/993893fc1d5c47b4a194340ad53ee8b9
I20251023 02:10:58.429803 23161 ts_tablet_manager.cc:1950] T 993893fc1d5c47b4a194340ad53ee8b9 P 7fb388975cb842e7b476da64bd50a992: Deleting consensus metadata
I20251023 02:10:58.430526 22612 catalog_manager.cc:4985] TS 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): tablet 993893fc1d5c47b4a194340ad53ee8b9 (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.510550 23115 tablet_service.cc:1552] Processing DeleteTablet for tablet 7bed03c029ee46d1b69acf63e259e57b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 02:10:58 UTC) from {username='slave'} at 127.0.0.1:60714
I20251023 02:10:58.510826 23161 tablet_replica.cc:333] stopping tablet replica
I20251023 02:10:58.510986 23161 raft_consensus.cc:2243] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:10:58.511106 23161 pending_rounds.cc:70] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Trying to abort 1 pending ops.
I20251023 02:10:58.511142 23161 pending_rounds.cc:77] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213815631929606144 op_type: NO_OP noop_request { }
I20251023 02:10:58.511229 23161 raft_consensus.cc:2889] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251023 02:10:58.511304 23161 raft_consensus.cc:2272] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:10:58.511906 23161 ts_tablet_manager.cc:1916] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 02:10:58.512810 22877 consensus_queue.cc:237] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:58.515841 22804 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:58.516315 22931 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:58.517079 23161 ts_tablet_manager.cc:1929] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 02:10:58.517243 23161 log.cc:1199] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/7bed03c029ee46d1b69acf63e259e57b
I20251023 02:10:58.517617 23161 ts_tablet_manager.cc:1950] T 7bed03c029ee46d1b69acf63e259e57b P 7fb388975cb842e7b476da64bd50a992: Deleting consensus metadata
I20251023 02:10:58.518651 22612 catalog_manager.cc:4985] TS 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): tablet 7bed03c029ee46d1b69acf63e259e57b (table dugtrio [id=ff8bcb6fa1c34ca891937062b146e48c]) successfully deleted
I20251023 02:10:58.519728 23163 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.520278 22804 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
W20251023 02:10:58.521102 22842 consensus_peers.cc:597] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b -> Peer c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): Couldn't send request to peer c3864b87aaa34f92b748ea4eff18d150. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b66d971101b4acdb207b7e7e36799d4. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:58.522303 23126 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:58.522711 22931 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:58.523799 23157 heartbeater.cc:499] Master 127.15.161.254:34399 was elected leader, sending a full tablet report...
I20251023 02:10:58.524680 22613 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6b66d971101b4acdb207b7e7e36799d4 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 6)
I20251023 02:10:58.524945 22627 catalog_manager.cc:5649] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.531256 23126 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.534874 23163 raft_consensus.cc:493] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 02:10:58.534979 23163 raft_consensus.cc:515] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:58.535344 23163 leader_election.cc:290] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:58.535786 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0" is_pre_election: true
I20251023 02:10:58.535785 23126 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992" is_pre_election: true
I20251023 02:10:58.535960 22804 raft_consensus.cc:2468] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 1.
I20251023 02:10:58.536005 23126 raft_consensus.cc:2410] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09b43b6f0a944b9096101927ba53a14b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 02:10:58.536293 22842 leader_election.cc:304] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0; no voters:
I20251023 02:10:58.536500 23163 raft_consensus.cc:2804] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 02:10:58.536577 23163 raft_consensus.cc:493] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 02:10:58.536643 23163 raft_consensus.cc:3060] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:58.538407 23163 raft_consensus.cc:515] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:58.538779 23163 leader_election.cc:290] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 2 election: Requested vote from peers 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447), 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:58.539134 22804 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e35508ce4004db383775ecd53a92ba0"
I20251023 02:10:58.539079 23126 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cea9618bd841449b8db98622d5a12e2d" candidate_uuid: "09b43b6f0a944b9096101927ba53a14b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb388975cb842e7b476da64bd50a992"
I20251023 02:10:58.539291 22804 raft_consensus.cc:3060] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:58.539311 23126 raft_consensus.cc:3060] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Advancing to term 2
I20251023 02:10:58.541504 22804 raft_consensus.cc:2468] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b43b6f0a944b9096101927ba53a14b in term 2.
I20251023 02:10:58.541581 23126 raft_consensus.cc:2410] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 09b43b6f0a944b9096101927ba53a14b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 02:10:58.541803 22842 leader_election.cc:304] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b43b6f0a944b9096101927ba53a14b, 6e35508ce4004db383775ecd53a92ba0; no voters:
I20251023 02:10:58.541967 23163 raft_consensus.cc:2804] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 2 FOLLOWER]: Leader election won for term 2
I20251023 02:10:58.542079 23163 raft_consensus.cc:697] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 2 LEADER]: Becoming Leader. State: Replica: 09b43b6f0a944b9096101927ba53a14b, State: Running, Role: LEADER
I20251023 02:10:58.542282 23163 consensus_queue.cc:237] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } }
I20251023 02:10:58.543990 22627 catalog_manager.cc:5649] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b reported cstate change: term changed from 1 to 2, leader changed from 7fb388975cb842e7b476da64bd50a992 (127.15.161.193) to 09b43b6f0a944b9096101927ba53a14b (127.15.161.195). New cstate: current_term: 2 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:58.562630 22987 consensus_queue.cc:579] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Leader has been unable to successfully communicate with peer 7fb388975cb842e7b476da64bd50a992 for more than 1 seconds (1.445s)
I20251023 02:10:58.565281 22804 consensus_queue.cc:237] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
W20251023 02:10:58.567116 22768 consensus_peers.cc:597] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 -> Peer c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): Couldn't send request to peer c3864b87aaa34f92b748ea4eff18d150. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a67fb103d57c4633a7955f574084feb3. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:58.567276 22877 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:58.567948 23125 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 02:10:58.568194 22987 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:58.568420 22943 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 02:10:58.570613 22943 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.571300 22877 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.573066 22612 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a67fb103d57c4633a7955f574084feb3 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 02:10:58.573674 23125 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.573971 22627 catalog_manager.cc:5649] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:58.577791 22804 consensus_queue.cc:237] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.579016 22877 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:58.579412 22928 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:58.579835 23125 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:58.580199 22928 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:58.580986 22943 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:58.581545 22877 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:58.582284 23125 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:58.583815 22612 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet a67fb103d57c4633a7955f574084feb3 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:58.584403 22625 catalog_manager.cc:5649] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:58.591251 23036 tablet_service.cc:1552] Processing DeleteTablet for tablet a67fb103d57c4633a7955f574084feb3 with delete_type TABLET_DATA_TOMBSTONED (TS c3864b87aaa34f92b748ea4eff18d150 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44774
W20251023 02:10:58.591945 22612 catalog_manager.cc:4952] TS c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): delete failed for tablet a67fb103d57c4633a7955f574084feb3 because the tablet was not found. No further retry: Not found: Tablet not found: a67fb103d57c4633a7955f574084feb3
I20251023 02:10:58.856010 23174 ts_tablet_manager.cc:933] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Initiating tablet copy from peer 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:58.856266 22987 consensus_queue.cc:1050] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:58.856549 23174 tablet_copy_client.cc:323] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:35611
I20251023 02:10:58.858207 23125 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:58.861989 22814 tablet_copy_service.cc:140] P 6e35508ce4004db383775ecd53a92ba0: Received BeginTabletCopySession request for tablet 75d3ebdb6f6e49829e45d892bb203c05 from peer c3864b87aaa34f92b748ea4eff18d150 ({username='slave'} at 127.0.0.1:46600)
I20251023 02:10:58.862093 22814 tablet_copy_service.cc:161] P 6e35508ce4004db383775ecd53a92ba0: Beginning new tablet copy session on tablet 75d3ebdb6f6e49829e45d892bb203c05 from peer c3864b87aaa34f92b748ea4eff18d150 at {username='slave'} at 127.0.0.1:46600: session id = c3864b87aaa34f92b748ea4eff18d150-75d3ebdb6f6e49829e45d892bb203c05
I20251023 02:10:58.864112 22814 tablet_copy_source_session.cc:215] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:58.865031 23174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75d3ebdb6f6e49829e45d892bb203c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:58.868662 23174 tablet_copy_client.cc:806] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:58.868867 23174 tablet_copy_client.cc:670] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:58.869930 23174 tablet_copy_client.cc:538] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:58.871835 23174 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap starting.
I20251023 02:10:58.875838 23174 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:58.876086 23174 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap complete.
I20251023 02:10:58.876253 23174 ts_tablet_manager.cc:1403] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20251023 02:10:58.876711 23174 raft_consensus.cc:359] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:58.876847 23174 raft_consensus.cc:740] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c3864b87aaa34f92b748ea4eff18d150, State: Initialized, Role: LEARNER
I20251023 02:10:58.876953 23174 consensus_queue.cc:260] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:58.877389 23071 heartbeater.cc:499] Master 127.15.161.254:34399 was elected leader, sending a full tablet report...
I20251023 02:10:58.877424 23174 ts_tablet_manager.cc:1434] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20251023 02:10:58.877911 22814 tablet_copy_service.cc:342] P 6e35508ce4004db383775ecd53a92ba0: Request end of tablet copy session c3864b87aaa34f92b748ea4eff18d150-75d3ebdb6f6e49829e45d892bb203c05 received from {username='slave'} at 127.0.0.1:46600
I20251023 02:10:58.878006 22814 tablet_copy_service.cc:434] P 6e35508ce4004db383775ecd53a92ba0: ending tablet copy session c3864b87aaa34f92b748ea4eff18d150-75d3ebdb6f6e49829e45d892bb203c05 on tablet 75d3ebdb6f6e49829e45d892bb203c05 with peer c3864b87aaa34f92b748ea4eff18d150
I20251023 02:10:58.924003 22804 raft_consensus.cc:1275] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 2 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251023 02:10:58.924451 23163 consensus_queue.cc:1048] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:58.927227 23125 raft_consensus.cc:1275] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 2 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20251023 02:10:58.927330 23125 pending_rounds.cc:85] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Aborting all ops after (but not including) 0
I20251023 02:10:58.927368 23125 pending_rounds.cc:107] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20251023 02:10:58.927416 23125 raft_consensus.cc:2889] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 02:10:58.927915 23163 consensus_queue.cc:1048] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 02:10:58.958680 22877 consensus_queue.cc:237] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:58.959861 23125 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:58.959901 22804 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:58.960208 23163 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:58.960317 23167 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:58.962013 22931 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:58.962512 22804 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:58.962636 23125 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:58.965330 22613 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 6b66d971101b4acdb207b7e7e36799d4 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:58.966635 22626 catalog_manager.cc:5649] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } }
I20251023 02:10:58.969254 23036 tablet_service.cc:1552] Processing DeleteTablet for tablet 6b66d971101b4acdb207b7e7e36799d4 with delete_type TABLET_DATA_TOMBSTONED (TS c3864b87aaa34f92b748ea4eff18d150 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44774
W20251023 02:10:58.969699 22612 catalog_manager.cc:4952] TS c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): delete failed for tablet 6b66d971101b4acdb207b7e7e36799d4 because the tablet was not found. No further retry: Not found: Tablet not found: 6b66d971101b4acdb207b7e7e36799d4
I20251023 02:10:59.316835 22804 consensus_queue.cc:237] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } }
I20251023 02:10:59.318188 22877 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:59.318207 23125 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 02:10:59.318598 23185 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:59.318771 22987 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 02:10:59.320422 22943 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:59.320855 22877 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:59.322829 22612 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 75d3ebdb6f6e49829e45d892bb203c05 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:10:59.323654 23125 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:59.324244 22625 catalog_manager.cc:5649] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: config changed from index 2 to 3, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) evicted. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } }
I20251023 02:10:59.326944 23036 tablet_service.cc:1552] Processing DeleteTablet for tablet 75d3ebdb6f6e49829e45d892bb203c05 with delete_type TABLET_DATA_TOMBSTONED (TS c3864b87aaa34f92b748ea4eff18d150 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44774
I20251023 02:10:59.327176 23173 tablet_replica.cc:333] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: stopping tablet replica
I20251023 02:10:59.327347 23173 raft_consensus.cc:2243] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Raft consensus shutting down.
I20251023 02:10:59.327495 23173 raft_consensus.cc:2272] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Raft consensus is shut down!
I20251023 02:10:59.328228 23173 ts_tablet_manager.cc:1916] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:10:59.331681 23173 ts_tablet_manager.cc:1929] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20251023 02:10:59.331763 23173 log.cc:1199] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-3-root/wals/75d3ebdb6f6e49829e45d892bb203c05
I20251023 02:10:59.332226 22612 catalog_manager.cc:4985] TS c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): tablet 75d3ebdb6f6e49829e45d892bb203c05 (table test-workload [id=b9fd5345e94e4278924be996247e46cd]) successfully deleted
I20251023 02:10:59.373091 22877 consensus_queue.cc:237] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.374713 22804 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:59.374778 23125 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251023 02:10:59.374810 22842 consensus_peers.cc:597] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b -> Peer c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): Couldn't send request to peer c3864b87aaa34f92b748ea4eff18d150. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b66d971101b4acdb207b7e7e36799d4. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:59.375149 23163 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:59.375346 23167 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:59.376976 22931 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.378603 22804 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.378585 23125 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.381026 22626 catalog_manager.cc:5649] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:59.390416 22804 consensus_queue.cc:237] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.391677 22877 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:59.391713 23125 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251023 02:10:59.391925 22768 consensus_peers.cc:597] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 -> Peer c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): Couldn't send request to peer c3864b87aaa34f92b748ea4eff18d150. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:59.392082 22987 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:59.392259 22943 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:59.394163 23185 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.394631 23125 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.397446 22877 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.397831 22627 catalog_manager.cc:5649] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:59.398478 22804 consensus_queue.cc:237] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.399832 22878 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:59.400127 23125 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 02:10:59.400171 23185 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 02:10:59.400473 22943 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20251023 02:10:59.401021 22768 consensus_peers.cc:597] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 -> Peer c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196:36489): Couldn't send request to peer c3864b87aaa34f92b748ea4eff18d150. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a67fb103d57c4633a7955f574084feb3. This is attempt 1: this message will repeat every 5th retry.
I20251023 02:10:59.403108 22943 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.403676 23126 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.404953 22878 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } } }
I20251023 02:10:59.407330 22626 catalog_manager.cc:5649] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index 3 to 4, NON_VOTER c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) added. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 02:10:59.833056 23189 ts_tablet_manager.cc:933] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Initiating tablet copy from peer 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:59.833670 23189 tablet_copy_client.cc:323] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:35611
I20251023 02:10:59.834259 22814 tablet_copy_service.cc:140] P 6e35508ce4004db383775ecd53a92ba0: Received BeginTabletCopySession request for tablet 75d3ebdb6f6e49829e45d892bb203c05 from peer c3864b87aaa34f92b748ea4eff18d150 ({username='slave'} at 127.0.0.1:46600)
I20251023 02:10:59.834408 22814 tablet_copy_service.cc:161] P 6e35508ce4004db383775ecd53a92ba0: Beginning new tablet copy session on tablet 75d3ebdb6f6e49829e45d892bb203c05 from peer c3864b87aaa34f92b748ea4eff18d150 at {username='slave'} at 127.0.0.1:46600: session id = c3864b87aaa34f92b748ea4eff18d150-75d3ebdb6f6e49829e45d892bb203c05
I20251023 02:10:59.836855 22814 tablet_copy_source_session.cc:215] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:59.837584 23189 ts_tablet_manager.cc:1916] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Deleting tablet data with delete state TABLET_DATA_COPYING
I20251023 02:10:59.844923 23189 ts_tablet_manager.cc:1929] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20251023 02:10:59.845302 23189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75d3ebdb6f6e49829e45d892bb203c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:59.846433 23191 ts_tablet_manager.cc:933] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: Initiating tablet copy from peer 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611)
I20251023 02:10:59.846963 23191 tablet_copy_client.cc:323] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.194:35611
I20251023 02:10:59.847467 22814 tablet_copy_service.cc:140] P 6e35508ce4004db383775ecd53a92ba0: Received BeginTabletCopySession request for tablet a67fb103d57c4633a7955f574084feb3 from peer c3864b87aaa34f92b748ea4eff18d150 ({username='slave'} at 127.0.0.1:46600)
I20251023 02:10:59.847575 22814 tablet_copy_service.cc:161] P 6e35508ce4004db383775ecd53a92ba0: Beginning new tablet copy session on tablet a67fb103d57c4633a7955f574084feb3 from peer c3864b87aaa34f92b748ea4eff18d150 at {username='slave'} at 127.0.0.1:46600: session id = c3864b87aaa34f92b748ea4eff18d150-a67fb103d57c4633a7955f574084feb3
I20251023 02:10:59.848173 23189 tablet_copy_client.cc:806] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:59.848450 23189 tablet_copy_client.cc:670] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:59.849633 23189 tablet_copy_client.cc:538] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:59.849716 22814 tablet_copy_source_session.cc:215] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:59.850955 23191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67fb103d57c4633a7955f574084feb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:59.852108 23189 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap starting.
I20251023 02:10:59.856338 23191 tablet_copy_client.cc:806] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:59.856539 23191 tablet_copy_client.cc:670] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:59.857817 23191 tablet_copy_client.cc:538] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:59.859081 23189 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:59.859386 23189 tablet_bootstrap.cc:492] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap complete.
I20251023 02:10:59.860827 23191 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap starting.
I20251023 02:10:59.866865 23191 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:59.867189 23191 tablet_bootstrap.cc:492] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap complete.
I20251023 02:10:59.867444 23191 ts_tablet_manager.cc:1403] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.003s
I20251023 02:10:59.867997 23191 raft_consensus.cc:359] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.868218 23191 raft_consensus.cc:740] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c3864b87aaa34f92b748ea4eff18d150, State: Initialized, Role: LEARNER
I20251023 02:10:59.868395 23191 consensus_queue.cc:260] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.868976 23191 ts_tablet_manager.cc:1434] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251023 02:10:59.869447 22814 tablet_copy_service.cc:342] P 6e35508ce4004db383775ecd53a92ba0: Request end of tablet copy session c3864b87aaa34f92b748ea4eff18d150-a67fb103d57c4633a7955f574084feb3 received from {username='slave'} at 127.0.0.1:46600
I20251023 02:10:59.869561 22814 tablet_copy_service.cc:434] P 6e35508ce4004db383775ecd53a92ba0: ending tablet copy session c3864b87aaa34f92b748ea4eff18d150-a67fb103d57c4633a7955f574084feb3 on tablet a67fb103d57c4633a7955f574084feb3 with peer c3864b87aaa34f92b748ea4eff18d150
I20251023 02:10:59.872160 23189 ts_tablet_manager.cc:1403] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Time spent bootstrapping tablet: real 0.020s user 0.020s sys 0.000s
I20251023 02:10:59.872633 23189 raft_consensus.cc:359] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.872761 23189 raft_consensus.cc:740] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c3864b87aaa34f92b748ea4eff18d150, State: Initialized, Role: LEARNER
I20251023 02:10:59.872923 23189 consensus_queue.cc:260] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.873346 23189 ts_tablet_manager.cc:1434] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 02:10:59.873960 22814 tablet_copy_service.cc:342] P 6e35508ce4004db383775ecd53a92ba0: Request end of tablet copy session c3864b87aaa34f92b748ea4eff18d150-75d3ebdb6f6e49829e45d892bb203c05 received from {username='slave'} at 127.0.0.1:46600
I20251023 02:10:59.874050 22814 tablet_copy_service.cc:434] P 6e35508ce4004db383775ecd53a92ba0: ending tablet copy session c3864b87aaa34f92b748ea4eff18d150-75d3ebdb6f6e49829e45d892bb203c05 on tablet 75d3ebdb6f6e49829e45d892bb203c05 with peer c3864b87aaa34f92b748ea4eff18d150
I20251023 02:10:59.968751 23189 ts_tablet_manager.cc:933] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: Initiating tablet copy from peer 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867)
I20251023 02:10:59.969216 23189 tablet_copy_client.cc:323] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Beginning tablet copy session from remote peer at address 127.15.161.195:34867
I20251023 02:10:59.974537 22888 tablet_copy_service.cc:140] P 09b43b6f0a944b9096101927ba53a14b: Received BeginTabletCopySession request for tablet 6b66d971101b4acdb207b7e7e36799d4 from peer c3864b87aaa34f92b748ea4eff18d150 ({username='slave'} at 127.0.0.1:54638)
I20251023 02:10:59.974637 22888 tablet_copy_service.cc:161] P 09b43b6f0a944b9096101927ba53a14b: Beginning new tablet copy session on tablet 6b66d971101b4acdb207b7e7e36799d4 from peer c3864b87aaa34f92b748ea4eff18d150 at {username='slave'} at 127.0.0.1:54638: session id = c3864b87aaa34f92b748ea4eff18d150-6b66d971101b4acdb207b7e7e36799d4
I20251023 02:10:59.976523 22888 tablet_copy_source_session.cc:215] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 02:10:59.977365 23189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b66d971101b4acdb207b7e7e36799d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 02:10:59.981616 23189 tablet_copy_client.cc:806] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 0 data blocks...
I20251023 02:10:59.981829 23189 tablet_copy_client.cc:670] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Starting download of 1 WAL segments...
I20251023 02:10:59.982970 23189 tablet_copy_client.cc:538] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 02:10:59.985441 23189 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap starting.
I20251023 02:10:59.990231 23189 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 02:10:59.990466 23189 tablet_bootstrap.cc:492] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: Bootstrap complete.
I20251023 02:10:59.990659 23189 ts_tablet_manager.cc:1403] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: Time spent bootstrapping tablet: real 0.005s user 0.007s sys 0.000s
I20251023 02:10:59.991146 23189 raft_consensus.cc:359] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.991266 23189 raft_consensus.cc:740] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c3864b87aaa34f92b748ea4eff18d150, State: Initialized, Role: LEARNER
I20251023 02:10:59.991394 23189 consensus_queue.cc:260] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: NON_VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: true } }
I20251023 02:10:59.991740 23189 ts_tablet_manager.cc:1434] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 02:10:59.992192 22888 tablet_copy_service.cc:342] P 09b43b6f0a944b9096101927ba53a14b: Request end of tablet copy session c3864b87aaa34f92b748ea4eff18d150-6b66d971101b4acdb207b7e7e36799d4 received from {username='slave'} at 127.0.0.1:54638
I20251023 02:10:59.992269 22888 tablet_copy_service.cc:434] P 09b43b6f0a944b9096101927ba53a14b: ending tablet copy session c3864b87aaa34f92b748ea4eff18d150-6b66d971101b4acdb207b7e7e36799d4 on tablet 6b66d971101b4acdb207b7e7e36799d4 with peer c3864b87aaa34f92b748ea4eff18d150
I20251023 02:11:00.217500 23046 raft_consensus.cc:1217] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251023 02:11:00.426949 23046 raft_consensus.cc:1217] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251023 02:11:00.518455 23046 raft_consensus.cc:1217] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251023 02:11:00.711414 23201 raft_consensus.cc:1064] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: attempting to promote NON_VOTER c3864b87aaa34f92b748ea4eff18d150 to VOTER
I20251023 02:11:00.712056 23201 consensus_queue.cc:237] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } }
I20251023 02:11:00.714428 22878 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:00.714452 23126 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:00.714428 23046 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:00.714932 23201 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:00.715126 23185 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:00.715354 23200 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:00.718219 23205 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.718787 23126 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.718787 23046 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.722616 22878 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.722945 22627 catalog_manager.cc:5649] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 reported cstate change: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } attrs { replace: true } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.732753 22804 consensus_queue.cc:237] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } }
I20251023 02:11:00.734182 23046 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 02:11:00.734377 22877 raft_consensus.cc:1275] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 02:11:00.734601 23185 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:11:00.734817 23201 consensus_queue.cc:1048] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:11:00.736827 23200 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7fb388975cb842e7b476da64bd50a992 (127.15.161.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.737279 22877 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7fb388975cb842e7b476da64bd50a992 (127.15.161.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.739709 22612 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 75d3ebdb6f6e49829e45d892bb203c05 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:11:00.740793 23046 raft_consensus.cc:2955] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7fb388975cb842e7b476da64bd50a992 (127.15.161.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.741192 22625 catalog_manager.cc:5649] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: config changed from index 5 to 6, VOTER 7fb388975cb842e7b476da64bd50a992 (127.15.161.193) evicted. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.744391 23115 tablet_service.cc:1552] Processing DeleteTablet for tablet 75d3ebdb6f6e49829e45d892bb203c05 with delete_type TABLET_DATA_TOMBSTONED (TS 7fb388975cb842e7b476da64bd50a992 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60714
I20251023 02:11:00.745186 23214 tablet_replica.cc:333] stopping tablet replica
I20251023 02:11:00.745371 23214 raft_consensus.cc:2243] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:00.745568 23214 raft_consensus.cc:2272] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:00.746227 23214 ts_tablet_manager.cc:1916] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:11:00.750196 23214 ts_tablet_manager.cc:1929] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251023 02:11:00.750317 23214 log.cc:1199] T 75d3ebdb6f6e49829e45d892bb203c05 P 7fb388975cb842e7b476da64bd50a992: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-0-root/wals/75d3ebdb6f6e49829e45d892bb203c05
I20251023 02:11:00.750885 22612 catalog_manager.cc:4985] TS 7fb388975cb842e7b476da64bd50a992 (127.15.161.193:38447): tablet 75d3ebdb6f6e49829e45d892bb203c05 (table test-workload [id=b9fd5345e94e4278924be996247e46cd]) successfully deleted
I20251023 02:11:00.935469 23201 raft_consensus.cc:1064] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: attempting to promote NON_VOTER c3864b87aaa34f92b748ea4eff18d150 to VOTER
I20251023 02:11:00.936056 23201 consensus_queue.cc:237] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } }
I20251023 02:11:00.937822 23046 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:00.937872 23126 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:00.938318 22877 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:00.938359 23185 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:00.938584 23201 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:00.938834 23200 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:00.941359 23185 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.941870 23126 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.941870 22877 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.945027 23046 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.946535 22626 catalog_manager.cc:5649] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:11:00.950184 22803 consensus_queue.cc:237] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } }
I20251023 02:11:00.951473 23046 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 02:11:00.952201 23205 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:11:00.952370 23126 raft_consensus.cc:1275] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 6e35508ce4004db383775ecd53a92ba0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 02:11:00.952863 23205 consensus_queue.cc:1048] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:11:00.954160 23185 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 09b43b6f0a944b9096101927ba53a14b (127.15.161.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.954718 23046 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 09b43b6f0a944b9096101927ba53a14b (127.15.161.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.957124 22612 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet a67fb103d57c4633a7955f574084feb3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:11:00.958431 22624 catalog_manager.cc:5649] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 reported cstate change: config changed from index 5 to 6, VOTER 09b43b6f0a944b9096101927ba53a14b (127.15.161.195) evicted. New cstate: current_term: 1 leader_uuid: "6e35508ce4004db383775ecd53a92ba0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:11:00.959724 23126 raft_consensus.cc:2955] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 09b43b6f0a944b9096101927ba53a14b (127.15.161.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:00.961570 22867 tablet_service.cc:1552] Processing DeleteTablet for tablet a67fb103d57c4633a7955f574084feb3 with delete_type TABLET_DATA_TOMBSTONED (TS 09b43b6f0a944b9096101927ba53a14b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54602
I20251023 02:11:00.962379 23219 tablet_replica.cc:333] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:11:00.962600 23219 raft_consensus.cc:2243] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:00.962924 23219 raft_consensus.cc:2272] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:00.964010 23219 ts_tablet_manager.cc:1916] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:11:00.968752 23219 ts_tablet_manager.cc:1929] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251023 02:11:00.968827 23219 log.cc:1199] T a67fb103d57c4633a7955f574084feb3 P 09b43b6f0a944b9096101927ba53a14b: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-2-root/wals/a67fb103d57c4633a7955f574084feb3
I20251023 02:11:00.969338 22613 catalog_manager.cc:4985] TS 09b43b6f0a944b9096101927ba53a14b (127.15.161.195:34867): tablet a67fb103d57c4633a7955f574084feb3 (table test-workload [id=b9fd5345e94e4278924be996247e46cd]) successfully deleted
I20251023 02:11:01.009964 23202 raft_consensus.cc:1064] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: attempting to promote NON_VOTER c3864b87aaa34f92b748ea4eff18d150 to VOTER
I20251023 02:11:01.010547 23202 consensus_queue.cc:237] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } }
I20251023 02:11:01.012231 23046 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 LEARNER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:01.012331 23126 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:01.012331 22804 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 02:11:01.012797 22931 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:01.013074 23203 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:01.013482 23203 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 02:11:01.016216 22931 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.016701 23046 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.020496 22627 catalog_manager.cc:5649] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 reported cstate change: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.027792 23126 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.027792 22804 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c3864b87aaa34f92b748ea4eff18d150 (127.15.161.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "6e35508ce4004db383775ecd53a92ba0" member_type: VOTER last_known_addr { host: "127.15.161.194" port: 35611 } attrs { replace: true } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.031718 22878 consensus_queue.cc:237] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } }
I20251023 02:11:01.032996 23046 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 02:11:01.033200 23126 raft_consensus.cc:1275] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Refusing update from remote peer 09b43b6f0a944b9096101927ba53a14b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 02:11:01.033457 22931 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:11:01.033666 23203 consensus_queue.cc:1048] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 02:11:01.035799 23221 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.035977 23046 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.036501 23126 raft_consensus.cc:2955] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } } }
I20251023 02:11:01.038421 22613 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 6b66d971101b4acdb207b7e7e36799d4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 02:11:01.039775 22624 catalog_manager.cc:5649] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b reported cstate change: config changed from index 5 to 6, VOTER 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194) evicted. New cstate: current_term: 1 leader_uuid: "09b43b6f0a944b9096101927ba53a14b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7fb388975cb842e7b476da64bd50a992" member_type: VOTER last_known_addr { host: "127.15.161.193" port: 38447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b43b6f0a944b9096101927ba53a14b" member_type: VOTER last_known_addr { host: "127.15.161.195" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3864b87aaa34f92b748ea4eff18d150" member_type: VOTER last_known_addr { host: "127.15.161.196" port: 36489 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 02:11:01.042608 22792 tablet_service.cc:1552] Processing DeleteTablet for tablet 6b66d971101b4acdb207b7e7e36799d4 with delete_type TABLET_DATA_TOMBSTONED (TS 6e35508ce4004db383775ecd53a92ba0 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:46556
I20251023 02:11:01.043141 23225 tablet_replica.cc:333] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:11:01.043322 23225 raft_consensus.cc:2243] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:01.043649 23225 raft_consensus.cc:2272] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:01.044528 23225 ts_tablet_manager.cc:1916] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 02:11:01.048146 23225 ts_tablet_manager.cc:1929] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251023 02:11:01.048231 23225 log.cc:1199] T 6b66d971101b4acdb207b7e7e36799d4 P 6e35508ce4004db383775ecd53a92ba0: Deleting WAL directory at /tmp/dist-test-taskj6dnA7/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761185276310797-16007-0/minicluster-data/ts-1-root/wals/6b66d971101b4acdb207b7e7e36799d4
I20251023 02:11:01.048717 22612 catalog_manager.cc:4985] TS 6e35508ce4004db383775ecd53a92ba0 (127.15.161.194:35611): tablet 6b66d971101b4acdb207b7e7e36799d4 (table test-workload [id=b9fd5345e94e4278924be996247e46cd]) successfully deleted
I20251023 02:11:03.405848 16007 tablet_server.cc:178] TabletServer@127.15.161.193:38447 shutting down...
I20251023 02:11:03.417742 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:11:03.418053 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:11:03.418258 16007 raft_consensus.cc:2243] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.418596 16007 raft_consensus.cc:2272] T a67fb103d57c4633a7955f574084feb3 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.419256 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:11:03.419412 16007 raft_consensus.cc:2243] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.419565 16007 raft_consensus.cc:2272] T cea9618bd841449b8db98622d5a12e2d P 7fb388975cb842e7b476da64bd50a992 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.420023 16007 tablet_replica.cc:333] stopping tablet replica
I20251023 02:11:03.420171 16007 raft_consensus.cc:2243] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.420406 16007 raft_consensus.cc:2272] T 6b66d971101b4acdb207b7e7e36799d4 P 7fb388975cb842e7b476da64bd50a992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.434808 16007 tablet_server.cc:195] TabletServer@127.15.161.193:38447 shutdown complete.
I20251023 02:11:03.438759 16007 tablet_server.cc:178] TabletServer@127.15.161.194:0 shutting down...
I20251023 02:11:03.457731 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:11:03.458007 16007 tablet_replica.cc:333] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:11:03.458195 16007 raft_consensus.cc:2243] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.458370 16007 raft_consensus.cc:2272] T cea9618bd841449b8db98622d5a12e2d P 6e35508ce4004db383775ecd53a92ba0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.458930 16007 tablet_replica.cc:333] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:11:03.459057 16007 raft_consensus.cc:2243] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:11:03.459311 16007 raft_consensus.cc:2272] T a67fb103d57c4633a7955f574084feb3 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.459757 16007 tablet_replica.cc:333] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0: stopping tablet replica
I20251023 02:11:03.459872 16007 raft_consensus.cc:2243] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:11:03.460084 16007 raft_consensus.cc:2272] T 75d3ebdb6f6e49829e45d892bb203c05 P 6e35508ce4004db383775ecd53a92ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.473994 16007 tablet_server.cc:195] TabletServer@127.15.161.194:0 shutdown complete.
I20251023 02:11:03.478000 16007 tablet_server.cc:178] TabletServer@127.15.161.195:0 shutting down...
I20251023 02:11:03.485845 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:11:03.486085 16007 tablet_replica.cc:333] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:11:03.486267 16007 raft_consensus.cc:2243] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.486522 16007 raft_consensus.cc:2272] T 75d3ebdb6f6e49829e45d892bb203c05 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.486971 16007 tablet_replica.cc:333] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:11:03.487064 16007 raft_consensus.cc:2243] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:11:03.487313 16007 raft_consensus.cc:2272] T 6b66d971101b4acdb207b7e7e36799d4 P 09b43b6f0a944b9096101927ba53a14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.487712 16007 tablet_replica.cc:333] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b: stopping tablet replica
I20251023 02:11:03.487799 16007 raft_consensus.cc:2243] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 2 LEADER]: Raft consensus shutting down.
I20251023 02:11:03.487977 16007 raft_consensus.cc:2272] T cea9618bd841449b8db98622d5a12e2d P 09b43b6f0a944b9096101927ba53a14b [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.501513 16007 tablet_server.cc:195] TabletServer@127.15.161.195:0 shutdown complete.
I20251023 02:11:03.505493 16007 tablet_server.cc:178] TabletServer@127.15.161.196:0 shutting down...
I20251023 02:11:03.513266 16007 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 02:11:03.513497 16007 tablet_replica.cc:333] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150: stopping tablet replica
I20251023 02:11:03.513651 16007 raft_consensus.cc:2243] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.513856 16007 raft_consensus.cc:2272] T 6b66d971101b4acdb207b7e7e36799d4 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.514393 16007 tablet_replica.cc:333] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150: stopping tablet replica
I20251023 02:11:03.514533 16007 raft_consensus.cc:2243] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.514724 16007 raft_consensus.cc:2272] T 75d3ebdb6f6e49829e45d892bb203c05 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.515128 16007 tablet_replica.cc:333] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150: stopping tablet replica
I20251023 02:11:03.515251 16007 raft_consensus.cc:2243] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 02:11:03.515379 16007 raft_consensus.cc:2272] T a67fb103d57c4633a7955f574084feb3 P c3864b87aaa34f92b748ea4eff18d150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.528262 16007 tablet_server.cc:195] TabletServer@127.15.161.196:0 shutdown complete.
I20251023 02:11:03.531217 16007 master.cc:561] Master@127.15.161.254:34399 shutting down...
I20251023 02:11:03.546363 16007 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 1 LEADER]: Raft consensus shutting down.
I20251023 02:11:03.546566 16007 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 02:11:03.546669 16007 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4675b1f40eb94dce84c3f101d95b6e8e: stopping tablet replica
I20251023 02:11:03.559636 16007 master.cc:583] Master@127.15.161.254:34399 shutdown complete.
[ OK ] AutoRebalancerTest.TestDeletedTables (8252 ms)
[----------] 14 tests from AutoRebalancerTest (187204 ms total)
[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (187204 ms total)
[ PASSED ] 13 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer
1 FAILED TEST
I20251023 02:11:03.572276 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251023 02:11:03.572350 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251023 02:11:03.572398 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:4712: suppressed but not reported on 12 messages since previous log ~5 seconds ago
I20251023 02:11:03.572438 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:5327: suppressed but not reported on 6 messages since previous log ~30 seconds ago
I20251023 02:11:03.572485 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 2 messages since previous log ~173 seconds ago
I20251023 02:11:03.572572 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 53 messages since previous log ~18 seconds ago
I20251023 02:11:03.572623 16007 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 28 messages since previous log ~8 seconds ago