[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from AutoRebalancerTest
[ RUN ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251023 20:15:06.959620 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:33727,127.15.81.253:46627,127.15.81.252:39533
I20251023 20:15:06.961650 15687 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251023 20:15:06.962865 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:06.973423 15692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:06.973484 15693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:06.973398 15695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:06.974123 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:06.974828 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:06.974996 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:06.975062 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250506975046 us; error 0 us; skew 500 ppm
I20251023 20:15:06.975332 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:06.977726 15687 webserver.cc:492] Webserver started at http://127.15.81.254:35861/ using document root <none> and password file <none>
I20251023 20:15:06.978355 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:06.978456 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:06.978775 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:06.981745 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "33e592fad1324c0f8be27198c9ab7c54"
format_stamp: "Formatted at 2025-10-23 20:15:06 on dist-test-slave-jr94"
I20251023 20:15:06.987259 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20251023 20:15:06.990247 15700 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:06.991581 15687 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20251023 20:15:06.991739 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "33e592fad1324c0f8be27198c9ab7c54"
format_stamp: "Formatted at 2025-10-23 20:15:06 on dist-test-slave-jr94"
I20251023 20:15:06.991909 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:07.007675 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:07.008397 15687 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251023 20:15:07.008649 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:07.027968 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:33727
I20251023 20:15:07.027990 15751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:33727 every 8 connection(s)
I20251023 20:15:07.030018 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:07.032547 15754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:07.032677 15755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.033879 15752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:07.038086 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:07.038334 15758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.038734 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:07.038820 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:07.038908 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250507038866 us; error 0 us; skew 500 ppm
I20251023 20:15:07.039070 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:07.040409 15687 webserver.cc:492] Webserver started at http://127.15.81.253:44669/ using document root <none> and password file <none>
I20251023 20:15:07.040623 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:07.040724 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:07.040966 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:07.041635 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root/instance:
uuid: "1e038b8feb76435ca5e47c50d9d72616"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.043408 15752 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:07.044091 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:07.045837 15762 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.046384 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:07.046499 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root
uuid: "1e038b8feb76435ca5e47c50d9d72616"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.046636 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:07.054113 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:07.054836 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:07.069046 15752 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } has no permanent_uuid. Determining permanent_uuid...
W20251023 20:15:07.070412 15702 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.253:46627: connect: Connection refused (error 111)
W20251023 20:15:07.072206 15752 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.81.253:46627: Network error: Client connection negotiation failed: client connection to 127.15.81.253:46627: connect: Connection refused (error 111)
I20251023 20:15:07.072686 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.253:46627
I20251023 20:15:07.072736 15816 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.253:46627 every 8 connection(s)
I20251023 20:15:07.074105 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 20:15:07.074175 15817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251023 20:15:07.076586 15819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:07.076913 15820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.077773 15817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } has no permanent_uuid. Determining permanent_uuid...
W20251023 20:15:07.083036 15823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.083065 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:07.083446 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:07.083520 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:07.083590 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250507083549 us; error 0 us; skew 500 ppm
I20251023 20:15:07.083763 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:07.085028 15687 webserver.cc:492] Webserver started at http://127.15.81.252:44679/ using document root <none> and password file <none>
I20251023 20:15:07.085266 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:07.085383 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:07.085567 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:07.086261 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root/instance:
uuid: "d50444625a0849cfba975a774355032a"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.087131 15817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:07.089293 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20251023 20:15:07.091094 15829 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.091486 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:15:07.091640 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root
uuid: "d50444625a0849cfba975a774355032a"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.091799 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:07.094376 15817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
W20251023 20:15:07.096402 15817 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.81.252:39533: Network error: Client connection negotiation failed: client connection to 127.15.81.252:39533: connect: Connection refused (error 111)
I20251023 20:15:07.106310 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:07.106868 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:07.120591 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.252:39533
I20251023 20:15:07.120656 15881 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.252:39533 every 8 connection(s)
I20251023 20:15:07.121762 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:15:07.121654 15752 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } attempt: 1
I20251023 20:15:07.122515 15882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:07.125064 15882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:07.127956 15752 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:07.132432 15882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:07.138477 15882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:07.140165 15752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: Bootstrap starting.
I20251023 20:15:07.144655 15752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:07.145999 15752 log.cc:826] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: Log is configured to *not* fsync() on all Append() calls
I20251023 20:15:07.146353 15882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: Bootstrap starting.
I20251023 20:15:07.147753 15882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:07.148574 15817 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } attempt: 1
I20251023 20:15:07.148813 15752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: No bootstrap required, opened a new log
I20251023 20:15:07.150209 15882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: No bootstrap required, opened a new log
I20251023 20:15:07.155643 15882 raft_consensus.cc:359] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.155644 15752 raft_consensus.cc:359] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.156075 15882 raft_consensus.cc:385] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:07.156101 15752 raft_consensus.cc:385] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:07.156258 15752 raft_consensus.cc:740] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e592fad1324c0f8be27198c9ab7c54, State: Initialized, Role: FOLLOWER
I20251023 20:15:07.156262 15882 raft_consensus.cc:740] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50444625a0849cfba975a774355032a, State: Initialized, Role: FOLLOWER
I20251023 20:15:07.156491 15817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: Bootstrap starting.
I20251023 20:15:07.157305 15882 consensus_queue.cc:260] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.157269 15752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.158208 15817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:07.158816 15889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.159662 15889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.159308 15888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.159873 15882 sys_catalog.cc:565] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:07.159950 15888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.159874 15752 sys_catalog.cc:565] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:07.161968 15817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: No bootstrap required, opened a new log
I20251023 20:15:07.162604 15817 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.162788 15817 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:07.162878 15817 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e038b8feb76435ca5e47c50d9d72616, State: Initialized, Role: FOLLOWER
I20251023 20:15:07.163120 15817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.163810 15893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.164109 15817 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:07.164110 15893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.174566 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251023 20:15:07.181226 15919 catalog_manager.cc:1568] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:15:07.181384 15919 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251023 20:15:07.182196 15912 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:15:07.182315 15912 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 20:15:07.188916 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251023 20:15:07.190006 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:07.190129 15925 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:15:07.190227 15925 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251023 20:15:07.194700 15927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:07.196507 15926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.198082 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:07.198225 15929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.198644 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:07.198767 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:07.198841 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250507198817 us; error 0 us; skew 500 ppm
I20251023 20:15:07.199045 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:07.200228 15888 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:07.200513 15687 webserver.cc:492] Webserver started at http://127.15.81.193:34221/ using document root <none> and password file <none>
I20251023 20:15:07.200505 15888 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.200870 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:07.200975 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:07.201138 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:07.202095 15888 leader_election.cc:290] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e038b8feb76435ca5e47c50d9d72616 (127.15.81.253:46627), d50444625a0849cfba975a774355032a (127.15.81.252:39533)
I20251023 20:15:07.202100 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "734935cd5f9c4c10ac9ac84583ce733c"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.202841 15857 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e592fad1324c0f8be27198c9ab7c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50444625a0849cfba975a774355032a" is_pre_election: true
I20251023 20:15:07.202814 15791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e592fad1324c0f8be27198c9ab7c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e038b8feb76435ca5e47c50d9d72616" is_pre_election: true
I20251023 20:15:07.203279 15857 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e592fad1324c0f8be27198c9ab7c54 in term 0.
I20251023 20:15:07.203281 15791 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e592fad1324c0f8be27198c9ab7c54 in term 0.
I20251023 20:15:07.203866 15702 leader_election.cc:304] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e038b8feb76435ca5e47c50d9d72616, 33e592fad1324c0f8be27198c9ab7c54; no voters:
I20251023 20:15:07.204231 15888 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:07.204381 15888 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:07.204478 15888 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:07.206007 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20251023 20:15:07.207002 15888 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.207916 15791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e592fad1324c0f8be27198c9ab7c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e038b8feb76435ca5e47c50d9d72616"
I20251023 20:15:07.207988 15857 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e592fad1324c0f8be27198c9ab7c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50444625a0849cfba975a774355032a"
I20251023 20:15:07.208115 15857 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:07.208120 15791 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:07.209036 15934 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.209555 15687 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20251023 20:15:07.209685 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "734935cd5f9c4c10ac9ac84583ce733c"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.209810 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:07.210067 15888 leader_election.cc:290] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [CANDIDATE]: Term 1 election: Requested vote from peers 1e038b8feb76435ca5e47c50d9d72616 (127.15.81.253:46627), d50444625a0849cfba975a774355032a (127.15.81.252:39533)
I20251023 20:15:07.213049 15857 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e592fad1324c0f8be27198c9ab7c54 in term 1.
I20251023 20:15:07.213438 15704 leader_election.cc:304] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e592fad1324c0f8be27198c9ab7c54, d50444625a0849cfba975a774355032a; no voters:
I20251023 20:15:07.213706 15888 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:07.214262 15791 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e592fad1324c0f8be27198c9ab7c54 in term 1.
I20251023 20:15:07.217115 15888 raft_consensus.cc:697] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 1 LEADER]: Becoming Leader. State: Replica: 33e592fad1324c0f8be27198c9ab7c54, State: Running, Role: LEADER
I20251023 20:15:07.217777 15888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } }
I20251023 20:15:07.221846 15939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e592fad1324c0f8be27198c9ab7c54. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.222148 15939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:07.223198 15941 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:15:07.225530 15941 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:15:07.233031 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:07.233840 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:07.234238 15857 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 1 FOLLOWER]: Refusing update from remote peer 33e592fad1324c0f8be27198c9ab7c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:07.234234 15791 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 1 FOLLOWER]: Refusing update from remote peer 33e592fad1324c0f8be27198c9ab7c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:07.234751 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:07.235239 15939 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:07.235464 15888 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:07.251488 15889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e592fad1324c0f8be27198c9ab7c54. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.251808 15889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.256323 15893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e592fad1324c0f8be27198c9ab7c54. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.261217 15893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.262411 15888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.263110 15888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:07.267016 15941 catalog_manager.cc:1357] Generated new cluster ID: 2675ae5b6b05472cb787155c42f825f9
I20251023 20:15:07.267143 15941 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:15:07.278584 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:07.278882 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251023 20:15:07.278910 15889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.279026 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:07.279207 15889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.279240 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.279476 15939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.280088 15939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:07.294637 15893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33e592fad1324c0f8be27198c9ab7c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e592fad1324c0f8be27198c9ab7c54" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 33727 } } peers { permanent_uuid: "1e038b8feb76435ca5e47c50d9d72616" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 46627 } } peers { permanent_uuid: "d50444625a0849cfba975a774355032a" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 39533 } } }
I20251023 20:15:07.294881 15893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:07.297981 15941 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:15:07.299168 15941 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:15:07.315588 15941 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: Generated new TSK 0
I20251023 20:15:07.317154 15941 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:15:07.338135 16008 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:37813 every 8 connection(s)
I20251023 20:15:07.338095 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:37813
I20251023 20:15:07.366194 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 20:15:07.369071 16009 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33727
I20251023 20:15:07.369423 16009 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.370347 16010 heartbeater.cc:344] Connected to a master server at 127.15.81.252:39533
I20251023 20:15:07.370488 16010 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.370667 16009 heartbeater.cc:507] Master 127.15.81.254:33727 requested a full tablet report, sending...
I20251023 20:15:07.371037 16010 heartbeater.cc:507] Master 127.15.81.252:39533 requested a full tablet report, sending...
I20251023 20:15:07.373327 15847 ts_manager.cc:194] Registered new tserver with Master: 734935cd5f9c4c10ac9ac84583ce733c (127.15.81.193:37813)
I20251023 20:15:07.373423 15717 ts_manager.cc:194] Registered new tserver with Master: 734935cd5f9c4c10ac9ac84583ce733c (127.15.81.193:37813)
I20251023 20:15:07.375373 16011 heartbeater.cc:344] Connected to a master server at 127.15.81.253:46627
I20251023 20:15:07.375535 16011 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.376122 16011 heartbeater.cc:507] Master 127.15.81.253:46627 requested a full tablet report, sending...
I20251023 20:15:07.376540 15717 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42582
I20251023 20:15:07.377384 15781 ts_manager.cc:194] Registered new tserver with Master: 734935cd5f9c4c10ac9ac84583ce733c (127.15.81.193:37813)
W20251023 20:15:07.377553 16016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:07.379540 16017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.382699 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:07.382956 16019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.383358 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:07.383421 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:07.383494 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250507383458 us; error 0 us; skew 500 ppm
I20251023 20:15:07.383656 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:07.385039 15687 webserver.cc:492] Webserver started at http://127.15.81.194:42799/ using document root <none> and password file <none>
I20251023 20:15:07.385263 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:07.385362 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:07.385505 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:07.386204 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "f22b926784e74a2985a899eede94c4cd"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.388909 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:07.390756 16024 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.391180 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 20:15:07.391304 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "f22b926784e74a2985a899eede94c4cd"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.391443 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:07.415781 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:07.416582 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:07.417316 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:07.418433 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:07.418529 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.418641 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:07.418705 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.437400 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:43869
I20251023 20:15:07.437438 16087 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:43869 every 8 connection(s)
I20251023 20:15:07.460372 16089 heartbeater.cc:344] Connected to a master server at 127.15.81.252:39533
I20251023 20:15:07.460578 16089 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.461220 16089 heartbeater.cc:507] Master 127.15.81.252:39533 requested a full tablet report, sending...
I20251023 20:15:07.462380 15847 ts_manager.cc:194] Registered new tserver with Master: f22b926784e74a2985a899eede94c4cd (127.15.81.194:43869)
I20251023 20:15:07.464170 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 20:15:07.467746 16088 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33727
I20251023 20:15:07.467887 16088 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.468539 16088 heartbeater.cc:507] Master 127.15.81.254:33727 requested a full tablet report, sending...
I20251023 20:15:07.469846 15717 ts_manager.cc:194] Registered new tserver with Master: f22b926784e74a2985a899eede94c4cd (127.15.81.194:43869)
I20251023 20:15:07.471190 15717 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42588
I20251023 20:15:07.473254 16090 heartbeater.cc:344] Connected to a master server at 127.15.81.253:46627
I20251023 20:15:07.473454 16090 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.474002 16090 heartbeater.cc:507] Master 127.15.81.253:46627 requested a full tablet report, sending...
W20251023 20:15:07.474040 16095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:07.474640 16096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.475231 15781 ts_manager.cc:194] Registered new tserver with Master: f22b926784e74a2985a899eede94c4cd (127.15.81.194:43869)
W20251023 20:15:07.476774 16098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:07.477170 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:07.477474 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:07.477576 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:07.477639 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250507477618 us; error 0 us; skew 500 ppm
I20251023 20:15:07.477830 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:07.479120 15687 webserver.cc:492] Webserver started at http://127.15.81.195:40319/ using document root <none> and password file <none>
I20251023 20:15:07.479398 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:07.479516 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:07.479665 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:07.480408 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "563f80e163b74666947e94a4da3856c0"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.483754 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251023 20:15:07.485956 16103 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.486382 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:07.486491 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "563f80e163b74666947e94a4da3856c0"
format_stamp: "Formatted at 2025-10-23 20:15:07 on dist-test-slave-jr94"
I20251023 20:15:07.486614 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:07.497985 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:07.498627 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:07.499228 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:07.500097 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:07.500190 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.500288 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:07.500385 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:07.520255 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:40909
I20251023 20:15:07.520294 16166 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:40909 every 8 connection(s)
I20251023 20:15:07.548885 16167 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33727
I20251023 20:15:07.549086 16167 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.550009 16167 heartbeater.cc:507] Master 127.15.81.254:33727 requested a full tablet report, sending...
I20251023 20:15:07.550889 15717 ts_manager.cc:194] Registered new tserver with Master: 563f80e163b74666947e94a4da3856c0 (127.15.81.195:40909)
I20251023 20:15:07.551806 15717 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42600
I20251023 20:15:07.557276 16169 heartbeater.cc:344] Connected to a master server at 127.15.81.253:46627
I20251023 20:15:07.557435 16169 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.557430 16168 heartbeater.cc:344] Connected to a master server at 127.15.81.252:39533
I20251023 20:15:07.557610 16168 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:07.557835 16169 heartbeater.cc:507] Master 127.15.81.253:46627 requested a full tablet report, sending...
I20251023 20:15:07.557961 16168 heartbeater.cc:507] Master 127.15.81.252:39533 requested a full tablet report, sending...
I20251023 20:15:07.558853 15847 ts_manager.cc:194] Registered new tserver with Master: 563f80e163b74666947e94a4da3856c0 (127.15.81.195:40909)
I20251023 20:15:07.558876 15781 ts_manager.cc:194] Registered new tserver with Master: 563f80e163b74666947e94a4da3856c0 (127.15.81.195:40909)
I20251023 20:15:07.559201 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.005814492s
I20251023 20:15:08.183086 15919 catalog_manager.cc:1269] Loaded cluster ID: 2675ae5b6b05472cb787155c42f825f9
I20251023 20:15:08.183202 15919 catalog_manager.cc:1562] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: loading cluster ID for follower catalog manager: success
I20251023 20:15:08.185240 15919 catalog_manager.cc:1584] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: acquiring CA information for follower catalog manager: success
I20251023 20:15:08.186406 15919 catalog_manager.cc:1612] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 20:15:08.191262 15925 catalog_manager.cc:1269] Loaded cluster ID: 2675ae5b6b05472cb787155c42f825f9
I20251023 20:15:08.191335 15925 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: loading cluster ID for follower catalog manager: success
I20251023 20:15:08.192984 15925 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: acquiring CA information for follower catalog manager: success
I20251023 20:15:08.193845 15925 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 20:15:08.379382 16009 heartbeater.cc:499] Master 127.15.81.254:33727 was elected leader, sending a full tablet report...
I20251023 20:15:08.472880 16088 heartbeater.cc:499] Master 127.15.81.254:33727 was elected leader, sending a full tablet report...
I20251023 20:15:08.553443 16167 heartbeater.cc:499] Master 127.15.81.254:33727 was elected leader, sending a full tablet report...
I20251023 20:15:09.173251 15909 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:09.614761 15687 test_util.cc:276] Using random seed: 385642107
I20251023 20:15:09.653817 15717 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42608:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251023 20:15:09.695309 16052 tablet_service.cc:1505] Processing CreateTablet for tablet 3858315d7e7c44cc9ef5ec2465624e97 (DEFAULT_TABLE table=test-workload [id=2cdf123faed345a69e4ceee131b01b24]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:15:09.696273 16052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3858315d7e7c44cc9ef5ec2465624e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:09.696426 16132 tablet_service.cc:1505] Processing CreateTablet for tablet ee0fa1a87fe14e1e8b43b0da2f178b31 (DEFAULT_TABLE table=test-workload [id=2cdf123faed345a69e4ceee131b01b24]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:15:09.696995 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0fa1a87fe14e1e8b43b0da2f178b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:09.697728 15973 tablet_service.cc:1505] Processing CreateTablet for tablet 0df82cc5b7d54375968ac23aecacd49e (DEFAULT_TABLE table=test-workload [id=2cdf123faed345a69e4ceee131b01b24]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:15:09.697938 15972 tablet_service.cc:1505] Processing CreateTablet for tablet d987b1008c254282b7253fe37ed87182 (DEFAULT_TABLE table=test-workload [id=2cdf123faed345a69e4ceee131b01b24]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:15:09.698383 15973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df82cc5b7d54375968ac23aecacd49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:09.698601 15972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d987b1008c254282b7253fe37ed87182. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:09.709605 16205 tablet_bootstrap.cc:492] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c: Bootstrap starting.
I20251023 20:15:09.710832 16203 tablet_bootstrap.cc:492] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd: Bootstrap starting.
I20251023 20:15:09.712225 16205 tablet_bootstrap.cc:654] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:09.713577 16203 tablet_bootstrap.cc:654] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:09.713928 16204 tablet_bootstrap.cc:492] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0: Bootstrap starting.
I20251023 20:15:09.716306 16204 tablet_bootstrap.cc:654] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:09.716321 16203 tablet_bootstrap.cc:492] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd: No bootstrap required, opened a new log
I20251023 20:15:09.716562 16203 ts_tablet_manager.cc:1403] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20251023 20:15:09.717010 16205 tablet_bootstrap.cc:492] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c: No bootstrap required, opened a new log
I20251023 20:15:09.717154 16205 ts_tablet_manager.cc:1403] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20251023 20:15:09.717381 16203 raft_consensus.cc:359] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22b926784e74a2985a899eede94c4cd" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43869 } }
I20251023 20:15:09.717535 16203 raft_consensus.cc:385] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:09.717641 16203 raft_consensus.cc:740] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22b926784e74a2985a899eede94c4cd, State: Initialized, Role: FOLLOWER
I20251023 20:15:09.717873 16203 consensus_queue.cc:260] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22b926784e74a2985a899eede94c4cd" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43869 } }
I20251023 20:15:09.718076 16203 raft_consensus.cc:399] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:09.718077 16205 raft_consensus.cc:359] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.718221 16203 raft_consensus.cc:493] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:09.718292 16205 raft_consensus.cc:385] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:09.718367 16203 raft_consensus.cc:3060] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:09.718461 16205 raft_consensus.cc:740] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734935cd5f9c4c10ac9ac84583ce733c, State: Initialized, Role: FOLLOWER
I20251023 20:15:09.718794 16205 consensus_queue.cc:260] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.719027 16205 raft_consensus.cc:399] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:09.719139 16205 raft_consensus.cc:493] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:09.719280 16205 raft_consensus.cc:3060] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:09.720439 16203 raft_consensus.cc:515] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22b926784e74a2985a899eede94c4cd" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43869 } }
I20251023 20:15:09.720662 16203 leader_election.cc:304] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f22b926784e74a2985a899eede94c4cd; no voters:
I20251023 20:15:09.722157 16205 raft_consensus.cc:515] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.722470 16204 tablet_bootstrap.cc:492] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0: No bootstrap required, opened a new log
I20251023 20:15:09.722465 16205 leader_election.cc:304] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 734935cd5f9c4c10ac9ac84583ce733c; no voters:
I20251023 20:15:09.722687 16204 ts_tablet_manager.cc:1403] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20251023 20:15:09.723605 16204 raft_consensus.cc:359] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563f80e163b74666947e94a4da3856c0" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 40909 } }
I20251023 20:15:09.723798 16204 raft_consensus.cc:385] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:09.723928 16204 raft_consensus.cc:740] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563f80e163b74666947e94a4da3856c0, State: Initialized, Role: FOLLOWER
I20251023 20:15:09.724189 16204 consensus_queue.cc:260] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563f80e163b74666947e94a4da3856c0" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 40909 } }
I20251023 20:15:09.724398 16204 raft_consensus.cc:399] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:09.724519 16204 raft_consensus.cc:493] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:09.724648 16204 raft_consensus.cc:3060] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:09.727365 16203 leader_election.cc:290] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:09.727517 16209 raft_consensus.cc:2804] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:09.728093 16204 raft_consensus.cc:515] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563f80e163b74666947e94a4da3856c0" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 40909 } }
I20251023 20:15:09.728405 16204 leader_election.cc:304] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 563f80e163b74666947e94a4da3856c0; no voters:
I20251023 20:15:09.733840 16204 leader_election.cc:290] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:09.734037 16214 raft_consensus.cc:2804] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:09.735458 16209 raft_consensus.cc:697] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 1 LEADER]: Becoming Leader. State: Replica: f22b926784e74a2985a899eede94c4cd, State: Running, Role: LEADER
I20251023 20:15:09.735945 16209 consensus_queue.cc:237] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22b926784e74a2985a899eede94c4cd" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43869 } }
I20251023 20:15:09.738777 16210 raft_consensus.cc:2804] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:09.743650 16210 raft_consensus.cc:697] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 1 LEADER]: Becoming Leader. State: Replica: 734935cd5f9c4c10ac9ac84583ce733c, State: Running, Role: LEADER
I20251023 20:15:09.743997 16210 consensus_queue.cc:237] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.744884 15717 catalog_manager.cc:5649] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd reported cstate change: term changed from 0 to 1, leader changed from <none> to f22b926784e74a2985a899eede94c4cd (127.15.81.194). New cstate: current_term: 1 leader_uuid: "f22b926784e74a2985a899eede94c4cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22b926784e74a2985a899eede94c4cd" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43869 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:09.747824 16203 ts_tablet_manager.cc:1434] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd: Time spent starting tablet: real 0.031s user 0.015s sys 0.015s
I20251023 20:15:09.748600 15716 catalog_manager.cc:5649] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c reported cstate change: term changed from 0 to 1, leader changed from <none> to 734935cd5f9c4c10ac9ac84583ce733c (127.15.81.193). New cstate: current_term: 1 leader_uuid: "734935cd5f9c4c10ac9ac84583ce733c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:09.738983 16205 leader_election.cc:290] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:09.741967 16204 ts_tablet_manager.cc:1434] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0: Time spent starting tablet: real 0.019s user 0.002s sys 0.015s
I20251023 20:15:09.742249 16214 raft_consensus.cc:697] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 1 LEADER]: Becoming Leader. State: Replica: 563f80e163b74666947e94a4da3856c0, State: Running, Role: LEADER
I20251023 20:15:09.761564 16214 consensus_queue.cc:237] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563f80e163b74666947e94a4da3856c0" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 40909 } }
I20251023 20:15:09.763090 16205 ts_tablet_manager.cc:1434] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c: Time spent starting tablet: real 0.046s user 0.007s sys 0.008s
I20251023 20:15:09.764369 16205 tablet_bootstrap.cc:492] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c: Bootstrap starting.
I20251023 20:15:09.765713 15717 catalog_manager.cc:5649] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 563f80e163b74666947e94a4da3856c0 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "563f80e163b74666947e94a4da3856c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563f80e163b74666947e94a4da3856c0" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 40909 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:09.766770 16205 tablet_bootstrap.cc:654] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:09.801416 16205 tablet_bootstrap.cc:492] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c: No bootstrap required, opened a new log
I20251023 20:15:09.801636 16205 ts_tablet_manager.cc:1403] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20251023 20:15:09.802294 16205 raft_consensus.cc:359] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.802464 16205 raft_consensus.cc:385] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:09.802547 16205 raft_consensus.cc:740] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734935cd5f9c4c10ac9ac84583ce733c, State: Initialized, Role: FOLLOWER
I20251023 20:15:09.802780 16205 consensus_queue.cc:260] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.802907 16205 raft_consensus.cc:399] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:09.802999 16205 raft_consensus.cc:493] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:09.803083 16205 raft_consensus.cc:3060] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:09.804939 16205 raft_consensus.cc:515] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.805135 16205 leader_election.cc:304] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 734935cd5f9c4c10ac9ac84583ce733c; no voters:
I20251023 20:15:09.805356 16205 leader_election.cc:290] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:09.805802 16205 ts_tablet_manager.cc:1434] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20251023 20:15:09.805830 16210 raft_consensus.cc:2804] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:09.806074 16210 raft_consensus.cc:697] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 1 LEADER]: Becoming Leader. State: Replica: 734935cd5f9c4c10ac9ac84583ce733c, State: Running, Role: LEADER
I20251023 20:15:09.806270 16210 consensus_queue.cc:237] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } }
I20251023 20:15:09.808432 15716 catalog_manager.cc:5649] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c reported cstate change: term changed from 0 to 1, leader changed from <none> to 734935cd5f9c4c10ac9ac84583ce733c (127.15.81.193). New cstate: current_term: 1 leader_uuid: "734935cd5f9c4c10ac9ac84583ce733c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "734935cd5f9c4c10ac9ac84583ce733c" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37813 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:09.836030 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:15:09.845868 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:09.846395 15687 tablet_replica.cc:333] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c: stopping tablet replica
I20251023 20:15:09.846843 15687 raft_consensus.cc:2243] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:09.847108 15687 raft_consensus.cc:2272] T d987b1008c254282b7253fe37ed87182 P 734935cd5f9c4c10ac9ac84583ce733c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.848693 15687 tablet_replica.cc:333] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c: stopping tablet replica
I20251023 20:15:09.848881 15687 raft_consensus.cc:2243] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:09.849009 15687 raft_consensus.cc:2272] T 0df82cc5b7d54375968ac23aecacd49e P 734935cd5f9c4c10ac9ac84583ce733c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.863972 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:15:09.868969 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:15:09.877974 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:09.878391 15687 tablet_replica.cc:333] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd: stopping tablet replica
I20251023 20:15:09.878588 15687 raft_consensus.cc:2243] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:09.878754 15687 raft_consensus.cc:2272] T 3858315d7e7c44cc9ef5ec2465624e97 P f22b926784e74a2985a899eede94c4cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.882768 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:15:09.885135 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:15:09.896941 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:09.897396 15687 tablet_replica.cc:333] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0: stopping tablet replica
I20251023 20:15:09.897617 15687 raft_consensus.cc:2243] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:09.897814 15687 raft_consensus.cc:2272] T ee0fa1a87fe14e1e8b43b0da2f178b31 P 563f80e163b74666947e94a4da3856c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.912480 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:15:09.915443 15687 master.cc:561] Master@127.15.81.254:33727 shutting down...
I20251023 20:15:09.923116 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:09.923508 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.923635 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 33e592fad1324c0f8be27198c9ab7c54: stopping tablet replica
I20251023 20:15:09.938400 15687 master.cc:583] Master@127.15.81.254:33727 shutdown complete.
I20251023 20:15:09.943212 15687 master.cc:561] Master@127.15.81.253:46627 shutting down...
I20251023 20:15:09.950071 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:09.950276 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.950380 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1e038b8feb76435ca5e47c50d9d72616: stopping tablet replica
I20251023 20:15:09.954077 15687 master.cc:583] Master@127.15.81.253:46627 shutdown complete.
I20251023 20:15:09.957463 15687 master.cc:561] Master@127.15.81.252:39533 shutting down...
I20251023 20:15:09.964277 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:09.964452 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:09.964598 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P d50444625a0849cfba975a774355032a: stopping tablet replica
I20251023 20:15:09.978039 15687 master.cc:583] Master@127.15.81.252:39533 shutdown complete.
[ OK ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing (3032 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn
I20251023 20:15:09.989810 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:33745
I20251023 20:15:09.990314 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:09.992426 16226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:09.992761 16227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:09.993093 16229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:09.993726 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:09.993975 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:09.994040 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:09.994096 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250509994065 us; error 0 us; skew 500 ppm
I20251023 20:15:09.994217 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:09.995292 15687 webserver.cc:492] Webserver started at http://127.15.81.254:46793/ using document root <none> and password file <none>
I20251023 20:15:09.995509 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:09.995569 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:09.995680 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:09.996238 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "255d9a20a205458c94f05b50910730d8"
format_stamp: "Formatted at 2025-10-23 20:15:09 on dist-test-slave-jr94"
I20251023 20:15:09.998624 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:15:10.000099 16234 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.000425 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:15:10.000579 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "255d9a20a205458c94f05b50910730d8"
format_stamp: "Formatted at 2025-10-23 20:15:09 on dist-test-slave-jr94"
I20251023 20:15:10.000687 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:10.021571 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:10.022130 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:10.036029 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:33745
I20251023 20:15:10.036062 16285 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:33745 every 8 connection(s)
I20251023 20:15:10.037410 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:10.041539 16286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8: Bootstrap starting.
I20251023 20:15:10.042806 16286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:10.044262 16286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8: No bootstrap required, opened a new log
I20251023 20:15:10.044836 16286 raft_consensus.cc:359] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "255d9a20a205458c94f05b50910730d8" member_type: VOTER }
I20251023 20:15:10.044999 16286 raft_consensus.cc:385] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:10.045090 16286 raft_consensus.cc:740] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255d9a20a205458c94f05b50910730d8, State: Initialized, Role: FOLLOWER
I20251023 20:15:10.045279 16286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "255d9a20a205458c94f05b50910730d8" member_type: VOTER }
I20251023 20:15:10.045439 16286 raft_consensus.cc:399] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:10.045517 16286 raft_consensus.cc:493] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:10.045636 16286 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:10.047276 16286 raft_consensus.cc:515] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "255d9a20a205458c94f05b50910730d8" member_type: VOTER }
I20251023 20:15:10.047470 16286 leader_election.cc:304] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 255d9a20a205458c94f05b50910730d8; no voters:
I20251023 20:15:10.047811 16286 leader_election.cc:290] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:10.047914 16289 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:10.048409 16289 raft_consensus.cc:697] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 1 LEADER]: Becoming Leader. State: Replica: 255d9a20a205458c94f05b50910730d8, State: Running, Role: LEADER
I20251023 20:15:10.048597 16286 sys_catalog.cc:565] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:10.048658 16289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "255d9a20a205458c94f05b50910730d8" member_type: VOTER }
I20251023 20:15:10.049607 16291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 255d9a20a205458c94f05b50910730d8. Latest consensus state: current_term: 1 leader_uuid: "255d9a20a205458c94f05b50910730d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "255d9a20a205458c94f05b50910730d8" member_type: VOTER } }
I20251023 20:15:10.049546 16290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "255d9a20a205458c94f05b50910730d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "255d9a20a205458c94f05b50910730d8" member_type: VOTER } }
I20251023 20:15:10.049762 16291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:10.049765 16290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:10.053287 16295 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:15:10.054612 16295 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:15:10.054947 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:15:10.057436 16295 catalog_manager.cc:1357] Generated new cluster ID: 69b04819bac744beb32abb673a360ac6
I20251023 20:15:10.057528 16295 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:15:10.065945 16295 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:15:10.066751 16295 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:15:10.077615 16295 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8: Generated new TSK 0
I20251023 20:15:10.077795 16295 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:15:10.087765 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:10.090391 16308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:10.090683 16309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:10.091490 16311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:10.091621 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:10.091917 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:10.091980 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:10.092028 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250510092001 us; error 0 us; skew 500 ppm
I20251023 20:15:10.092159 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:10.093338 15687 webserver.cc:492] Webserver started at http://127.15.81.193:33917/ using document root <none> and password file <none>
I20251023 20:15:10.093540 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:10.093621 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:10.093740 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:10.094318 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "c1b215e0869a4027834da3a6dcfea3e6"
format_stamp: "Formatted at 2025-10-23 20:15:10 on dist-test-slave-jr94"
I20251023 20:15:10.096661 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:10.098119 16316 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.098441 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:10.098542 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "c1b215e0869a4027834da3a6dcfea3e6"
format_stamp: "Formatted at 2025-10-23 20:15:10 on dist-test-slave-jr94"
I20251023 20:15:10.098665 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:10.109102 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:10.109611 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:10.110100 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:10.110916 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:10.110985 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.111097 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:10.111152 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.125715 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:35787
I20251023 20:15:10.125744 16379 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:35787 every 8 connection(s)
I20251023 20:15:10.127377 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:10.130280 16384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:10.131243 16385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:10.131356 16387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:10.132241 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:10.132505 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:10.132593 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:10.132658 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250510132634 us; error 0 us; skew 500 ppm
I20251023 20:15:10.132828 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:10.134209 15687 webserver.cc:492] Webserver started at http://127.15.81.194:34339/ using document root <none> and password file <none>
I20251023 20:15:10.134524 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:10.134635 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:10.134809 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:10.135457 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "e36e6090c1534a159466eba7a367b06a"
format_stamp: "Formatted at 2025-10-23 20:15:10 on dist-test-slave-jr94"
I20251023 20:15:10.136389 16380 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33745
I20251023 20:15:10.136569 16380 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:10.137027 16380 heartbeater.cc:507] Master 127.15.81.254:33745 requested a full tablet report, sending...
I20251023 20:15:10.137863 16251 ts_manager.cc:194] Registered new tserver with Master: c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787)
I20251023 20:15:10.138509 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251023 20:15:10.139074 16251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47314
I20251023 20:15:10.140300 16392 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.140605 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 20:15:10.140728 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "e36e6090c1534a159466eba7a367b06a"
format_stamp: "Formatted at 2025-10-23 20:15:10 on dist-test-slave-jr94"
I20251023 20:15:10.140861 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:10.147344 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:10.147841 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:10.148336 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:10.149142 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:10.149240 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.149313 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:10.149366 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.164027 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:41165
I20251023 20:15:10.164062 16454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:41165 every 8 connection(s)
I20251023 20:15:10.165665 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:10.168623 16458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:10.168648 16459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:10.172441 16455 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33745
I20251023 20:15:10.172569 16455 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:10.172959 16455 heartbeater.cc:507] Master 127.15.81.254:33745 requested a full tablet report, sending...
I20251023 20:15:10.173821 16251 ts_manager.cc:194] Registered new tserver with Master: e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165)
I20251023 20:15:10.174826 16251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47316
I20251023 20:15:10.175519 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:10.175637 16462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:10.176086 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:10.176159 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:10.176234 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250510176200 us; error 0 us; skew 500 ppm
I20251023 20:15:10.176368 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:10.177464 15687 webserver.cc:492] Webserver started at http://127.15.81.195:34219/ using document root <none> and password file <none>
I20251023 20:15:10.177663 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:10.177753 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:10.177868 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:10.178455 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
format_stamp: "Formatted at 2025-10-23 20:15:10 on dist-test-slave-jr94"
I20251023 20:15:10.180883 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:15:10.182359 16466 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.182696 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:15:10.182793 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
format_stamp: "Formatted at 2025-10-23 20:15:10 on dist-test-slave-jr94"
I20251023 20:15:10.182956 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:10.212992 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:10.213582 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:10.214128 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:10.214934 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:10.214994 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.215098 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:10.215145 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:10.227659 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:37183
I20251023 20:15:10.227692 16528 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:37183 every 8 connection(s)
I20251023 20:15:10.233991 16529 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33745
I20251023 20:15:10.234138 16529 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:10.234517 16529 heartbeater.cc:507] Master 127.15.81.254:33745 requested a full tablet report, sending...
I20251023 20:15:10.235311 16251 ts_manager.cc:194] Registered new tserver with Master: 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:10.235323 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006811855s
I20251023 20:15:10.236066 16251 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47318
I20251023 20:15:11.140344 16380 heartbeater.cc:499] Master 127.15.81.254:33745 was elected leader, sending a full tablet report...
I20251023 20:15:11.176291 16455 heartbeater.cc:499] Master 127.15.81.254:33745 was elected leader, sending a full tablet report...
I20251023 20:15:11.237237 16529 heartbeater.cc:499] Master 127.15.81.254:33745 was elected leader, sending a full tablet report...
I20251023 20:15:11.262596 15687 test_util.cc:276] Using random seed: 387289948
I20251023 20:15:11.274535 16251 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47322:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o""\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:15:11.275228 16251 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:15:11.312983 16345 tablet_service.cc:1505] Processing CreateTablet for tablet 7b25ea079ba24e5a8f9caa3a868b77fa (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251023 20:15:11.313392 16342 tablet_service.cc:1505] Processing CreateTablet for tablet cdcad6626a79486d944c3c177b6564bd (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251023 20:15:11.312610 16344 tablet_service.cc:1505] Processing CreateTablet for tablet f00a0736570c4022964eb3d775a116d2 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251023 20:15:11.314055 16343 tablet_service.cc:1505] Processing CreateTablet for tablet d862a5c73b9242429e1974df9e40e99b (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251023 20:15:11.313601 16345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b25ea079ba24e5a8f9caa3a868b77fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.314723 16344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00a0736570c4022964eb3d775a116d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.315227 16341 tablet_service.cc:1505] Processing CreateTablet for tablet 9866d8ae70484eaaa2766c24d15ebc76 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251023 20:15:11.315769 16341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9866d8ae70484eaaa2766c24d15ebc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.319077 16343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d862a5c73b9242429e1974df9e40e99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.321002 16342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcad6626a79486d944c3c177b6564bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.321461 16494 tablet_service.cc:1505] Processing CreateTablet for tablet 7b25ea079ba24e5a8f9caa3a868b77fa (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251023 20:15:11.322382 16494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b25ea079ba24e5a8f9caa3a868b77fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.323221 16493 tablet_service.cc:1505] Processing CreateTablet for tablet f00a0736570c4022964eb3d775a116d2 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251023 20:15:11.324087 16493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00a0736570c4022964eb3d775a116d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.325039 16339 tablet_service.cc:1505] Processing CreateTablet for tablet caa0acadb3814c2c9b3237325cba46ac (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251023 20:15:11.325795 16340 tablet_service.cc:1505] Processing CreateTablet for tablet 8963a2dc524541858760687a81b55cbf (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251023 20:15:11.326579 16339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa0acadb3814c2c9b3237325cba46ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.326609 16492 tablet_service.cc:1505] Processing CreateTablet for tablet d862a5c73b9242429e1974df9e40e99b (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251023 20:15:11.329226 16491 tablet_service.cc:1505] Processing CreateTablet for tablet cdcad6626a79486d944c3c177b6564bd (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251023 20:15:11.330278 16340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8963a2dc524541858760687a81b55cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.329497 16338 tablet_service.cc:1505] Processing CreateTablet for tablet 8eb6f20483e14b9fb5d019c70e804459 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251023 20:15:11.331326 16491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcad6626a79486d944c3c177b6564bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.332149 16489 tablet_service.cc:1505] Processing CreateTablet for tablet 8963a2dc524541858760687a81b55cbf (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251023 20:15:11.331923 16490 tablet_service.cc:1505] Processing CreateTablet for tablet 9866d8ae70484eaaa2766c24d15ebc76 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251023 20:15:11.333382 16492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d862a5c73b9242429e1974df9e40e99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.333400 16487 tablet_service.cc:1505] Processing CreateTablet for tablet 8eb6f20483e14b9fb5d019c70e804459 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251023 20:15:11.335186 16488 tablet_service.cc:1505] Processing CreateTablet for tablet caa0acadb3814c2c9b3237325cba46ac (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251023 20:15:11.337770 16489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8963a2dc524541858760687a81b55cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.343856 16338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb6f20483e14b9fb5d019c70e804459. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.348516 16490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9866d8ae70484eaaa2766c24d15ebc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.348766 16488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa0acadb3814c2c9b3237325cba46ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.349102 16487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb6f20483e14b9fb5d019c70e804459. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.366822 16418 tablet_service.cc:1505] Processing CreateTablet for tablet d862a5c73b9242429e1974df9e40e99b (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251023 20:15:11.366796 16419 tablet_service.cc:1505] Processing CreateTablet for tablet f00a0736570c4022964eb3d775a116d2 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251023 20:15:11.367141 16417 tablet_service.cc:1505] Processing CreateTablet for tablet cdcad6626a79486d944c3c177b6564bd (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251023 20:15:11.367573 16415 tablet_service.cc:1505] Processing CreateTablet for tablet 8963a2dc524541858760687a81b55cbf (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251023 20:15:11.367671 16416 tablet_service.cc:1505] Processing CreateTablet for tablet 9866d8ae70484eaaa2766c24d15ebc76 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251023 20:15:11.369494 16418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d862a5c73b9242429e1974df9e40e99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.370501 16415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8963a2dc524541858760687a81b55cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.371695 16420 tablet_service.cc:1505] Processing CreateTablet for tablet 7b25ea079ba24e5a8f9caa3a868b77fa (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251023 20:15:11.372737 16550 tablet_bootstrap.cc:492] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.372943 16414 tablet_service.cc:1505] Processing CreateTablet for tablet caa0acadb3814c2c9b3237325cba46ac (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251023 20:15:11.374276 16416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9866d8ae70484eaaa2766c24d15ebc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.377072 16419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00a0736570c4022964eb3d775a116d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.377069 16413 tablet_service.cc:1505] Processing CreateTablet for tablet 8eb6f20483e14b9fb5d019c70e804459 (DEFAULT_TABLE table=test-workload [id=1a0858204b204933a8ce47de2a940fb7]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251023 20:15:11.377324 16414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caa0acadb3814c2c9b3237325cba46ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.377904 16417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcad6626a79486d944c3c177b6564bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.378103 16420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b25ea079ba24e5a8f9caa3a868b77fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.378582 16549 tablet_bootstrap.cc:492] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.378894 16413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb6f20483e14b9fb5d019c70e804459. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:11.406605 16549 tablet_bootstrap.cc:654] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.411027 16551 tablet_bootstrap.cc:492] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.415797 16549 tablet_bootstrap.cc:492] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.415966 16549 ts_tablet_manager.cc:1403] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.047s user 0.000s sys 0.007s
I20251023 20:15:11.416985 16549 raft_consensus.cc:359] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.417212 16550 tablet_bootstrap.cc:654] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.417248 16549 raft_consensus.cc:385] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.417356 16549 raft_consensus.cc:740] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.417630 16549 consensus_queue.cc:260] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.418259 16551 tablet_bootstrap.cc:654] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.420038 16551 tablet_bootstrap.cc:492] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.420167 16551 ts_tablet_manager.cc:1403] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20251023 20:15:11.420739 16551 raft_consensus.cc:359] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.420912 16551 raft_consensus.cc:385] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.421010 16551 raft_consensus.cc:740] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.421204 16551 consensus_queue.cc:260] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.423141 16549 ts_tablet_manager.cc:1434] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20251023 20:15:11.423497 16549 tablet_bootstrap.cc:492] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.423528 16551 ts_tablet_manager.cc:1434] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20251023 20:15:11.424049 16551 tablet_bootstrap.cc:492] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.424659 16550 tablet_bootstrap.cc:492] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.424836 16550 ts_tablet_manager.cc:1403] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.060s user 0.000s sys 0.008s
I20251023 20:15:11.425714 16550 raft_consensus.cc:359] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.425940 16550 raft_consensus.cc:385] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.426056 16550 raft_consensus.cc:740] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.426297 16550 consensus_queue.cc:260] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.427004 16551 tablet_bootstrap.cc:654] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.427059 16549 tablet_bootstrap.cc:654] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.428303 16550 ts_tablet_manager.cc:1434] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251023 20:15:11.428637 16550 tablet_bootstrap.cc:492] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.430749 16550 tablet_bootstrap.cc:654] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.433905 16551 tablet_bootstrap.cc:492] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.434047 16551 ts_tablet_manager.cc:1403] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20251023 20:15:11.434672 16551 raft_consensus.cc:359] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.434844 16551 raft_consensus.cc:385] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.434928 16551 raft_consensus.cc:740] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.435146 16551 consensus_queue.cc:260] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.435587 16551 ts_tablet_manager.cc:1434] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:15:11.435632 16549 tablet_bootstrap.cc:492] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.435825 16549 ts_tablet_manager.cc:1403] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20251023 20:15:11.435889 16551 tablet_bootstrap.cc:492] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.436815 16549 raft_consensus.cc:359] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.437100 16549 raft_consensus.cc:385] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.437405 16549 raft_consensus.cc:740] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.437667 16551 tablet_bootstrap.cc:654] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.437804 16549 consensus_queue.cc:260] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.438871 16550 tablet_bootstrap.cc:492] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.439028 16550 ts_tablet_manager.cc:1403] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20251023 20:15:11.439549 16549 ts_tablet_manager.cc:1434] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:15:11.439852 16549 tablet_bootstrap.cc:492] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.439801 16550 raft_consensus.cc:359] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.439990 16550 raft_consensus.cc:385] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.440079 16550 raft_consensus.cc:740] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.440337 16550 consensus_queue.cc:260] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.440800 16550 ts_tablet_manager.cc:1434] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:15:11.441548 16550 tablet_bootstrap.cc:492] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.441692 16549 tablet_bootstrap.cc:654] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.443490 16550 tablet_bootstrap.cc:654] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.445746 16549 tablet_bootstrap.cc:492] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.445848 16551 tablet_bootstrap.cc:492] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.445887 16549 ts_tablet_manager.cc:1403] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20251023 20:15:11.445991 16551 ts_tablet_manager.cc:1403] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20251023 20:15:11.446131 16550 tablet_bootstrap.cc:492] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.446269 16550 ts_tablet_manager.cc:1403] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:15:11.446708 16549 raft_consensus.cc:359] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.446808 16551 raft_consensus.cc:359] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.446913 16549 raft_consensus.cc:385] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.447013 16551 raft_consensus.cc:385] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.446962 16550 raft_consensus.cc:359] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.447021 16549 raft_consensus.cc:740] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.447121 16550 raft_consensus.cc:385] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.447125 16551 raft_consensus.cc:740] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.447228 16550 raft_consensus.cc:740] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.447441 16551 consensus_queue.cc:260] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.447504 16550 consensus_queue.cc:260] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.447471 16549 consensus_queue.cc:260] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.448019 16550 ts_tablet_manager.cc:1434] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:15:11.448074 16549 ts_tablet_manager.cc:1434] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20251023 20:15:11.448307 16550 tablet_bootstrap.cc:492] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.448338 16549 tablet_bootstrap.cc:492] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.448391 16551 ts_tablet_manager.cc:1434] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:15:11.448655 16551 tablet_bootstrap.cc:492] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.450209 16550 tablet_bootstrap.cc:654] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.450330 16549 tablet_bootstrap.cc:654] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.450537 16551 tablet_bootstrap.cc:654] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.461486 16550 tablet_bootstrap.cc:492] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.461670 16550 ts_tablet_manager.cc:1403] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.002s
I20251023 20:15:11.461766 16549 tablet_bootstrap.cc:492] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.461931 16549 ts_tablet_manager.cc:1403] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.002s
I20251023 20:15:11.462328 16551 tablet_bootstrap.cc:492] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.462531 16551 ts_tablet_manager.cc:1403] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.011s
I20251023 20:15:11.462574 16550 raft_consensus.cc:359] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.462579 16549 raft_consensus.cc:359] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.462822 16550 raft_consensus.cc:385] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.462869 16549 raft_consensus.cc:385] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.462916 16550 raft_consensus.cc:740] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.462972 16549 raft_consensus.cc:740] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.463163 16550 consensus_queue.cc:260] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.463243 16549 consensus_queue.cc:260] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.463289 16551 raft_consensus.cc:359] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.463455 16551 raft_consensus.cc:385] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.463558 16551 raft_consensus.cc:740] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.463810 16549 ts_tablet_manager.cc:1434] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:15:11.463806 16551 consensus_queue.cc:260] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.464205 16549 tablet_bootstrap.cc:492] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.465340 16551 ts_tablet_manager.cc:1434] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251023 20:15:11.465657 16550 ts_tablet_manager.cc:1434] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:15:11.465663 16551 tablet_bootstrap.cc:492] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.465966 16549 tablet_bootstrap.cc:654] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.466868 16550 tablet_bootstrap.cc:492] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.467432 16551 tablet_bootstrap.cc:654] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.468598 16550 tablet_bootstrap.cc:654] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.473783 16550 tablet_bootstrap.cc:492] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.473949 16550 ts_tablet_manager.cc:1403] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20251023 20:15:11.474824 16550 raft_consensus.cc:359] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.475039 16550 raft_consensus.cc:385] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.475148 16550 raft_consensus.cc:740] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.475443 16550 consensus_queue.cc:260] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.476135 16550 ts_tablet_manager.cc:1434] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 20:15:11.476475 16550 tablet_bootstrap.cc:492] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.477800 16549 tablet_bootstrap.cc:492] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.477938 16549 ts_tablet_manager.cc:1403] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.002s
I20251023 20:15:11.478632 16549 raft_consensus.cc:359] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.478755 16550 tablet_bootstrap.cc:654] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.478850 16549 raft_consensus.cc:385] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.478948 16549 raft_consensus.cc:740] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.479213 16549 consensus_queue.cc:260] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
W20251023 20:15:11.480051 16530 tablet.cc:2378] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 20:15:11.480285 16549 ts_tablet_manager.cc:1434] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:15:11.480553 16549 tablet_bootstrap.cc:492] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.481715 16551 tablet_bootstrap.cc:492] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.481894 16551 ts_tablet_manager.cc:1403] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20251023 20:15:11.482833 16549 tablet_bootstrap.cc:654] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.482936 16551 raft_consensus.cc:359] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.483251 16551 raft_consensus.cc:385] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.483361 16551 raft_consensus.cc:740] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.483711 16551 consensus_queue.cc:260] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.486804 16551 ts_tablet_manager.cc:1434] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20251023 20:15:11.487073 16551 tablet_bootstrap.cc:492] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.487962 16550 tablet_bootstrap.cc:492] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.488126 16550 ts_tablet_manager.cc:1403] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20251023 20:15:11.488775 16551 tablet_bootstrap.cc:654] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.488808 16550 raft_consensus.cc:359] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.489013 16550 raft_consensus.cc:385] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.489116 16550 raft_consensus.cc:740] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.489408 16550 consensus_queue.cc:260] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.490459 16550 ts_tablet_manager.cc:1434] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:15:11.490768 16550 tablet_bootstrap.cc:492] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.490820 16549 tablet_bootstrap.cc:492] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.490976 16549 ts_tablet_manager.cc:1403] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20251023 20:15:11.491308 16551 tablet_bootstrap.cc:492] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.491516 16551 ts_tablet_manager.cc:1403] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:15:11.491719 16549 raft_consensus.cc:359] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.491928 16549 raft_consensus.cc:385] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.492015 16549 raft_consensus.cc:740] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.492309 16549 consensus_queue.cc:260] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.492537 16550 tablet_bootstrap.cc:654] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.492540 16551 raft_consensus.cc:359] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.492862 16551 raft_consensus.cc:385] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.493000 16551 raft_consensus.cc:740] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.493355 16551 consensus_queue.cc:260] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.493851 16551 ts_tablet_manager.cc:1434] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:15:11.494202 16551 tablet_bootstrap.cc:492] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.494776 16549 ts_tablet_manager.cc:1434] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251023 20:15:11.495901 16549 tablet_bootstrap.cc:492] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.496054 16551 tablet_bootstrap.cc:654] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.497974 16549 tablet_bootstrap.cc:654] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.500577 16550 tablet_bootstrap.cc:492] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.500766 16550 ts_tablet_manager.cc:1403] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20251023 20:15:11.501648 16550 raft_consensus.cc:359] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.501823 16550 raft_consensus.cc:385] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.501943 16550 raft_consensus.cc:740] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.502043 16551 tablet_bootstrap.cc:492] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.502085 16549 tablet_bootstrap.cc:492] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.502213 16551 ts_tablet_manager.cc:1403] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20251023 20:15:11.502216 16549 ts_tablet_manager.cc:1403] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20251023 20:15:11.502228 16550 consensus_queue.cc:260] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.502689 16550 ts_tablet_manager.cc:1434] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 20:15:11.502923 16550 tablet_bootstrap.cc:492] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Bootstrap starting.
I20251023 20:15:11.503026 16549 raft_consensus.cc:359] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.503204 16549 raft_consensus.cc:385] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.503305 16549 raft_consensus.cc:740] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.503552 16549 consensus_queue.cc:260] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.503993 16549 ts_tablet_manager.cc:1434] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 20:15:11.504230 16549 tablet_bootstrap.cc:492] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6: Bootstrap starting.
I20251023 20:15:11.504516 16550 tablet_bootstrap.cc:654] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.505939 16549 tablet_bootstrap.cc:654] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.506430 16550 tablet_bootstrap.cc:492] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5: No bootstrap required, opened a new log
I20251023 20:15:11.506569 16550 ts_tablet_manager.cc:1403] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:15:11.507230 16550 raft_consensus.cc:359] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.507411 16550 raft_consensus.cc:385] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.507494 16550 raft_consensus.cc:740] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.507678 16551 raft_consensus.cc:359] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.507718 16550 consensus_queue.cc:260] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.507887 16557 raft_consensus.cc:493] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.507998 16551 raft_consensus.cc:385] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.508096 16551 raft_consensus.cc:740] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.508095 16557 raft_consensus.cc:515] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.508373 16551 consensus_queue.cc:260] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.509181 16557 leader_election.cc:290] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165)
I20251023 20:15:11.510093 16551 ts_tablet_manager.cc:1434] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251023 20:15:11.510370 16551 tablet_bootstrap.cc:492] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a: Bootstrap starting.
I20251023 20:15:11.512387 16551 tablet_bootstrap.cc:654] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:11.514323 16549 tablet_bootstrap.cc:492] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6: No bootstrap required, opened a new log
I20251023 20:15:11.514482 16549 ts_tablet_manager.cc:1403] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.007s
I20251023 20:15:11.515031 16549 raft_consensus.cc:359] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.515177 16549 raft_consensus.cc:385] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.515259 16549 raft_consensus.cc:740] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.515507 16549 consensus_queue.cc:260] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.515952 16549 ts_tablet_manager.cc:1434] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:15:11.516345 16557 raft_consensus.cc:493] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.516516 16557 raft_consensus.cc:515] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.517109 16557 leader_election.cc:290] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165)
I20251023 20:15:11.521232 16550 ts_tablet_manager.cc:1434] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20251023 20:15:11.524624 16551 tablet_bootstrap.cc:492] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a: No bootstrap required, opened a new log
I20251023 20:15:11.524859 16551 ts_tablet_manager.cc:1403] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20251023 20:15:11.526473 16551 raft_consensus.cc:359] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.526719 16551 raft_consensus.cc:385] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:11.526808 16551 raft_consensus.cc:740] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Initialized, Role: FOLLOWER
I20251023 20:15:11.527071 16551 consensus_queue.cc:260] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.527686 16551 ts_tablet_manager.cc:1434] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:15:11.528950 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d862a5c73b9242429e1974df9e40e99b" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6" is_pre_election: true
I20251023 20:15:11.529031 16354 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caa0acadb3814c2c9b3237325cba46ac" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6" is_pre_election: true
I20251023 20:15:11.529311 16355 raft_consensus.cc:2468] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 0.
I20251023 20:15:11.529433 16354 raft_consensus.cc:2468] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 0.
I20251023 20:15:11.529798 16469 leader_election.cc:304] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, c1b215e0869a4027834da3a6dcfea3e6; no voters:
I20251023 20:15:11.530055 16557 raft_consensus.cc:2804] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.530092 16469 leader_election.cc:304] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, c1b215e0869a4027834da3a6dcfea3e6; no voters:
I20251023 20:15:11.530157 16557 raft_consensus.cc:493] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.530282 16557 raft_consensus.cc:3060] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.530884 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d862a5c73b9242429e1974df9e40e99b" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a" is_pre_election: true
I20251023 20:15:11.531185 16430 raft_consensus.cc:2468] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 0.
I20251023 20:15:11.531544 16429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caa0acadb3814c2c9b3237325cba46ac" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a" is_pre_election: true
I20251023 20:15:11.531749 16429 raft_consensus.cc:2468] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 0.
I20251023 20:15:11.533169 16581 raft_consensus.cc:2804] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.533277 16581 raft_consensus.cc:493] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.533416 16581 raft_consensus.cc:3060] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.533994 16557 raft_consensus.cc:515] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.534577 16557 leader_election.cc:290] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 election: Requested vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165)
I20251023 20:15:11.534826 16354 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d862a5c73b9242429e1974df9e40e99b" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
I20251023 20:15:11.535001 16354 raft_consensus.cc:3060] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.534976 16429 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d862a5c73b9242429e1974df9e40e99b" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a"
I20251023 20:15:11.535156 16429 raft_consensus.cc:3060] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.535813 16581 raft_consensus.cc:515] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.536343 16581 leader_election.cc:290] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 election: Requested vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165)
I20251023 20:15:11.536989 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caa0acadb3814c2c9b3237325cba46ac" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a"
I20251023 20:15:11.537159 16430 raft_consensus.cc:3060] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.538303 16429 raft_consensus.cc:2468] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 1.
I20251023 20:15:11.538329 16354 raft_consensus.cc:2468] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 1.
I20251023 20:15:11.538699 16470 leader_election.cc:304] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.538722 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caa0acadb3814c2c9b3237325cba46ac" candidate_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
I20251023 20:15:11.538937 16355 raft_consensus.cc:3060] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.539114 16557 raft_consensus.cc:2804] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.539288 16557 raft_consensus.cc:697] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 LEADER]: Becoming Leader. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Running, Role: LEADER
I20251023 20:15:11.539530 16430 raft_consensus.cc:2468] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 1.
I20251023 20:15:11.539549 16557 consensus_queue.cc:237] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.539969 16470 leader_election.cc:304] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.540230 16581 raft_consensus.cc:2804] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.540820 16581 raft_consensus.cc:697] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 LEADER]: Becoming Leader. State: Replica: 3a3ba6bf43894c9fa6e62c2c3932bfd5, State: Running, Role: LEADER
I20251023 20:15:11.541152 16581 consensus_queue.cc:237] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.542650 16355 raft_consensus.cc:2468] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a3ba6bf43894c9fa6e62c2c3932bfd5 in term 1.
I20251023 20:15:11.544445 16251 catalog_manager.cc:5649] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:11.544991 16251 catalog_manager.cc:5649] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:11.561354 16556 raft_consensus.cc:493] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.561530 16556 raft_consensus.cc:515] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.562332 16556 leader_election.cc:290] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.567500 16553 raft_consensus.cc:493] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.567693 16553 raft_consensus.cc:515] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.568825 16553 leader_election.cc:290] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.569667 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b25ea079ba24e5a8f9caa3a868b77fa" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6" is_pre_election: true
I20251023 20:15:11.570493 16355 raft_consensus.cc:2468] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.570277 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b25ea079ba24e5a8f9caa3a868b77fa" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" is_pre_election: true
I20251023 20:15:11.570662 16504 raft_consensus.cc:2468] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.570938 16395 leader_election.cc:304] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.571242 16556 raft_consensus.cc:2804] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.571370 16556 raft_consensus.cc:493] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.571470 16556 raft_consensus.cc:3060] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.574698 16556 raft_consensus.cc:515] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.575580 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b25ea079ba24e5a8f9caa3a868b77fa" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:11.575590 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7b25ea079ba24e5a8f9caa3a868b77fa" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
I20251023 20:15:11.575770 16504 raft_consensus.cc:3060] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.575816 16355 raft_consensus.cc:3060] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.578513 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdcad6626a79486d944c3c177b6564bd" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a" is_pre_election: true
I20251023 20:15:11.578728 16430 raft_consensus.cc:2468] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 0.
I20251023 20:15:11.578830 16556 leader_election.cc:290] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Requested vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.578768 16355 raft_consensus.cc:2468] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.579162 16320 leader_election.cc:304] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.579342 16395 leader_election.cc:304] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.578768 16504 raft_consensus.cc:2468] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.579416 16553 raft_consensus.cc:2804] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.579288 16503 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdcad6626a79486d944c3c177b6564bd" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" is_pre_election: true
I20251023 20:15:11.579564 16553 raft_consensus.cc:493] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.579608 16503 raft_consensus.cc:2468] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 0.
I20251023 20:15:11.579670 16553 raft_consensus.cc:3060] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.579728 16556 raft_consensus.cc:2804] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.580179 16556 raft_consensus.cc:697] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Becoming Leader. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Running, Role: LEADER
I20251023 20:15:11.580451 16556 consensus_queue.cc:237] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.581885 16553 raft_consensus.cc:515] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.582497 16553 leader_election.cc:290] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 election: Requested vote from peers e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.582974 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdcad6626a79486d944c3c177b6564bd" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a"
I20251023 20:15:11.583194 16430 raft_consensus.cc:3060] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.583171 16503 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdcad6626a79486d944c3c177b6564bd" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:11.583360 16503 raft_consensus.cc:3060] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.583663 16251 catalog_manager.cc:5649] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a reported cstate change: term changed from 0 to 1, leader changed from <none> to e36e6090c1534a159466eba7a367b06a (127.15.81.194). New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:11.585593 16430 raft_consensus.cc:2468] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 1.
I20251023 20:15:11.586009 16320 leader_election.cc:304] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.586265 16553 raft_consensus.cc:2804] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.586378 16503 raft_consensus.cc:2468] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 1.
I20251023 20:15:11.589097 16553 raft_consensus.cc:697] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 1 LEADER]: Becoming Leader. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Running, Role: LEADER
I20251023 20:15:11.589370 16553 consensus_queue.cc:237] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.591404 16251 catalog_manager.cc:5649] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "c1b215e0869a4027834da3a6dcfea3e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:11.627247 16556 raft_consensus.cc:493] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.627450 16556 raft_consensus.cc:515] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.628067 16556 leader_election.cc:290] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.628571 16503 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" is_pre_election: true
I20251023 20:15:11.628569 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6" is_pre_election: true
I20251023 20:15:11.628808 16355 raft_consensus.cc:2468] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.628814 16503 raft_consensus.cc:2468] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.629211 16395 leader_election.cc:304] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.629452 16556 raft_consensus.cc:2804] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.629540 16556 raft_consensus.cc:493] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.629627 16556 raft_consensus.cc:3060] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.631359 16553 raft_consensus.cc:493] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.631464 16556 raft_consensus.cc:515] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.631541 16553 raft_consensus.cc:515] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.632055 16556 leader_election.cc:290] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Requested vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.632411 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
I20251023 20:15:11.632421 16553 leader_election.cc:290] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.632547 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8eb6f20483e14b9fb5d019c70e804459" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a" is_pre_election: true
I20251023 20:15:11.632627 16355 raft_consensus.cc:3060] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.632726 16430 raft_consensus.cc:2468] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 0.
I20251023 20:15:11.632969 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8eb6f20483e14b9fb5d019c70e804459" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" is_pre_election: true
I20251023 20:15:11.633073 16320 leader_election.cc:304] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.633152 16504 raft_consensus.cc:2468] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 0.
I20251023 20:15:11.633476 16553 raft_consensus.cc:2804] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.633611 16553 raft_consensus.cc:493] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.633721 16553 raft_consensus.cc:3060] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.634905 16355 raft_consensus.cc:2468] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.632402 16503 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:11.635226 16503 raft_consensus.cc:3060] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.635270 16395 leader_election.cc:304] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.635530 16556 raft_consensus.cc:2804] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.635749 16556 raft_consensus.cc:697] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Becoming Leader. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Running, Role: LEADER
I20251023 20:15:11.635780 16553 raft_consensus.cc:515] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.636050 16556 consensus_queue.cc:237] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.636538 16553 leader_election.cc:290] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 election: Requested vote from peers e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.636994 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8eb6f20483e14b9fb5d019c70e804459" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:11.637068 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8eb6f20483e14b9fb5d019c70e804459" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e6090c1534a159466eba7a367b06a"
I20251023 20:15:11.637236 16504 raft_consensus.cc:3060] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.638005 16503 raft_consensus.cc:2468] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.639386 16504 raft_consensus.cc:2468] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 1.
I20251023 20:15:11.639719 16317 leader_election.cc:304] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, c1b215e0869a4027834da3a6dcfea3e6; no voters:
I20251023 20:15:11.639954 16553 raft_consensus.cc:2804] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.637224 16430 raft_consensus.cc:3060] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.640159 16553 raft_consensus.cc:697] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 LEADER]: Becoming Leader. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Running, Role: LEADER
I20251023 20:15:11.640440 16553 consensus_queue.cc:237] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.641182 16251 catalog_manager.cc:5649] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a reported cstate change: term changed from 0 to 1, leader changed from <none> to e36e6090c1534a159466eba7a367b06a (127.15.81.194). New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:11.642949 16430 raft_consensus.cc:2468] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 1.
I20251023 20:15:11.643596 16249 catalog_manager.cc:5649] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "c1b215e0869a4027834da3a6dcfea3e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:11.724937 16556 raft_consensus.cc:493] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.725143 16556 raft_consensus.cc:515] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.725768 16556 leader_election.cc:290] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.726282 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8963a2dc524541858760687a81b55cbf" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6" is_pre_election: true
I20251023 20:15:11.726297 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8963a2dc524541858760687a81b55cbf" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" is_pre_election: true
I20251023 20:15:11.726475 16355 raft_consensus.cc:2468] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.726522 16504 raft_consensus.cc:2468] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.726903 16395 leader_election.cc:304] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.727139 16556 raft_consensus.cc:2804] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.727259 16556 raft_consensus.cc:493] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.727366 16556 raft_consensus.cc:3060] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.729315 16556 raft_consensus.cc:515] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.729849 16556 leader_election.cc:290] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Requested vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.730257 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8963a2dc524541858760687a81b55cbf" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:11.730250 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8963a2dc524541858760687a81b55cbf" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
I20251023 20:15:11.730453 16504 raft_consensus.cc:3060] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.730463 16355 raft_consensus.cc:3060] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.733284 16355 raft_consensus.cc:2468] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.733292 16504 raft_consensus.cc:2468] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.733693 16393 leader_election.cc:304] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.733963 16556 raft_consensus.cc:2804] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.734089 16556 raft_consensus.cc:697] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Becoming Leader. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Running, Role: LEADER
I20251023 20:15:11.734349 16556 consensus_queue.cc:237] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.736855 16249 catalog_manager.cc:5649] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a reported cstate change: term changed from 0 to 1, leader changed from <none> to e36e6090c1534a159466eba7a367b06a (127.15.81.194). New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:11.745060 16556 raft_consensus.cc:493] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:11.745203 16556 raft_consensus.cc:515] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.745661 16556 leader_election.cc:290] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.746098 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f00a0736570c4022964eb3d775a116d2" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6" is_pre_election: true
I20251023 20:15:11.746091 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f00a0736570c4022964eb3d775a116d2" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" is_pre_election: true
I20251023 20:15:11.746318 16355 raft_consensus.cc:2468] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.746341 16504 raft_consensus.cc:2468] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 0.
I20251023 20:15:11.746764 16393 leader_election.cc:304] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.747020 16556 raft_consensus.cc:2804] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:11.747095 16556 raft_consensus.cc:493] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:11.747192 16556 raft_consensus.cc:3060] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.748925 16556 raft_consensus.cc:515] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.749403 16556 leader_election.cc:290] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Requested vote from peers c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193:35787), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:11.749707 16355 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f00a0736570c4022964eb3d775a116d2" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
I20251023 20:15:11.749776 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f00a0736570c4022964eb3d775a116d2" candidate_uuid: "e36e6090c1534a159466eba7a367b06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:11.749886 16355 raft_consensus.cc:3060] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.750010 16504 raft_consensus.cc:3060] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:11.751837 16355 raft_consensus.cc:2468] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.751928 16504 raft_consensus.cc:2468] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36e6090c1534a159466eba7a367b06a in term 1.
I20251023 20:15:11.752204 16395 leader_election.cc:304] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b215e0869a4027834da3a6dcfea3e6, e36e6090c1534a159466eba7a367b06a; no voters:
I20251023 20:15:11.752547 16556 raft_consensus.cc:2804] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:11.752728 16556 raft_consensus.cc:697] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Becoming Leader. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Running, Role: LEADER
I20251023 20:15:11.753008 16556 consensus_queue.cc:237] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:11.755295 16249 catalog_manager.cc:5649] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a reported cstate change: term changed from 0 to 1, leader changed from <none> to e36e6090c1534a159466eba7a367b06a (127.15.81.194). New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:11.854813 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:11.857409 16597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:11.857594 16598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:11.858217 16600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:11.858732 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:11.858985 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:11.859063 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:11.859118 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250511859102 us; error 0 us; skew 500 ppm
I20251023 20:15:11.859285 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:11.860422 15687 webserver.cc:492] Webserver started at http://127.15.81.196:46715/ using document root <none> and password file <none>
I20251023 20:15:11.860658 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:11.860744 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:11.860931 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:11.861600 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "4a90097b2af84557940df6589435b5af"
format_stamp: "Formatted at 2025-10-23 20:15:11 on dist-test-slave-jr94"
I20251023 20:15:11.864096 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 20:15:11.865705 16605 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:11.866057 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:11.866176 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "4a90097b2af84557940df6589435b5af"
format_stamp: "Formatted at 2025-10-23 20:15:11 on dist-test-slave-jr94"
I20251023 20:15:11.866289 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:11.892191 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:11.892722 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:11.895532 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:11.896746 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:11.896862 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:11.896981 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:11.897055 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:11.913935 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:43695
I20251023 20:15:11.913983 16667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:43695 every 8 connection(s)
I20251023 20:15:11.921723 16668 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33745
I20251023 20:15:11.921865 16668 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:11.922318 16668 heartbeater.cc:507] Master 127.15.81.254:33745 requested a full tablet report, sending...
I20251023 20:15:11.922400 16557 consensus_queue.cc:1048] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:11.923218 16249 ts_manager.cc:194] Registered new tserver with Master: 4a90097b2af84557940df6589435b5af (127.15.81.196:43695)
I20251023 20:15:11.924292 16249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47330
I20251023 20:15:11.927496 16581 consensus_queue.cc:1048] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.037066 16556 consensus_queue.cc:1048] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.041750 16589 consensus_queue.cc:1048] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.044325 16581 consensus_queue.cc:1048] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.048166 16582 consensus_queue.cc:1048] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.054129 16305 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:12.061996 16430 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76"
dest_uuid: "e36e6090c1534a159466eba7a367b06a"
mode: GRACEFUL
new_leader_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
from {username='slave'} at 127.0.0.1:44790
I20251023 20:15:12.062225 16430 raft_consensus.cc:606] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Received request to transfer leadership to TS c1b215e0869a4027834da3a6dcfea3e6
I20251023 20:15:12.062719 16305 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:15:12.062969 16305 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:12.101913 16591 consensus_queue.cc:1048] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.105505 16591 consensus_queue.cc:1048] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.109133 16553 consensus_queue.cc:1048] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.109364 16685 consensus_queue.cc:1048] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.136512 16589 consensus_queue.cc:1048] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.139930 16556 consensus_queue.cc:1048] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.141418 16556 consensus_queue.cc:1048] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.146147 16556 consensus_queue.cc:1048] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.149673 16589 raft_consensus.cc:993] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: : Instructing follower c1b215e0869a4027834da3a6dcfea3e6 to start an election
I20251023 20:15:12.149870 16589 raft_consensus.cc:1081] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Signalling peer c1b215e0869a4027834da3a6dcfea3e6 to start an election
I20251023 20:15:12.150673 16354 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76"
dest_uuid: "c1b215e0869a4027834da3a6dcfea3e6"
from {username='slave'} at 127.0.0.1:35956
I20251023 20:15:12.150920 16354 raft_consensus.cc:493] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:15:12.151062 16354 raft_consensus.cc:3060] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:12.152938 16354 raft_consensus.cc:515] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:12.153425 16354 leader_election.cc:290] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 2 election: Requested vote from peers e36e6090c1534a159466eba7a367b06a (127.15.81.194:41165), 3a3ba6bf43894c9fa6e62c2c3932bfd5 (127.15.81.195:37183)
I20251023 20:15:12.153733 16430 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e36e6090c1534a159466eba7a367b06a"
I20251023 20:15:12.153865 16504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9866d8ae70484eaaa2766c24d15ebc76" candidate_uuid: "c1b215e0869a4027834da3a6dcfea3e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5"
I20251023 20:15:12.153915 16430 raft_consensus.cc:3055] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:15:12.154031 16430 raft_consensus.cc:740] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e36e6090c1534a159466eba7a367b06a, State: Running, Role: LEADER
I20251023 20:15:12.154040 16504 raft_consensus.cc:3060] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:12.154331 16430 consensus_queue.cc:260] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:12.154621 16430 raft_consensus.cc:3060] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:12.155984 16504 raft_consensus.cc:2468] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 2.
I20251023 20:15:12.156296 16317 leader_election.cc:304] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3ba6bf43894c9fa6e62c2c3932bfd5, c1b215e0869a4027834da3a6dcfea3e6; no voters:
I20251023 20:15:12.156550 16685 raft_consensus.cc:2804] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:15:12.156538 16430 raft_consensus.cc:2468] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b215e0869a4027834da3a6dcfea3e6 in term 2.
I20251023 20:15:12.156759 16685 raft_consensus.cc:697] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 2 LEADER]: Becoming Leader. State: Replica: c1b215e0869a4027834da3a6dcfea3e6, State: Running, Role: LEADER
I20251023 20:15:12.157014 16685 consensus_queue.cc:237] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } }
I20251023 20:15:12.158687 16251 catalog_manager.cc:5649] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 reported cstate change: term changed from 1 to 2, leader changed from e36e6090c1534a159466eba7a367b06a (127.15.81.194) to c1b215e0869a4027834da3a6dcfea3e6 (127.15.81.193). New cstate: current_term: 2 leader_uuid: "c1b215e0869a4027834da3a6dcfea3e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:12.165762 16589 consensus_queue.cc:1048] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.169348 16589 consensus_queue.cc:1048] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:12.621260 16430 raft_consensus.cc:1275] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 2 FOLLOWER]: Refusing update from remote peer c1b215e0869a4027834da3a6dcfea3e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:15:12.621829 16685 consensus_queue.cc:1048] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:12.624488 16504 raft_consensus.cc:1275] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 2 FOLLOWER]: Refusing update from remote peer c1b215e0869a4027834da3a6dcfea3e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:15:12.625442 16685 consensus_queue.cc:1048] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:12.925535 16668 heartbeater.cc:499] Master 127.15.81.254:33745 was elected leader, sending a full tablet report...
I20251023 20:15:14.063387 16305 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:14.064759 16305 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:14.064941 16305 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:16.065315 16305 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:16.066708 16305 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:16.066882 16305 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:17.069984 16504 consensus_queue.cc:237] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } attrs { replace: true } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } }
I20251023 20:15:17.072641 16354 raft_consensus.cc:1275] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Refusing update from remote peer 3a3ba6bf43894c9fa6e62c2c3932bfd5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.073258 16582 consensus_queue.cc:1048] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.073648 16430 raft_consensus.cc:1275] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Refusing update from remote peer 3a3ba6bf43894c9fa6e62c2c3932bfd5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.074263 16582 consensus_queue.cc:1048] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.077661 16582 raft_consensus.cc:2955] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } attrs { replace: true } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.078024 16354 raft_consensus.cc:2955] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } attrs { replace: true } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
W20251023 20:15:17.081974 16468 consensus_peers.cc:597] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 -> Peer 4a90097b2af84557940df6589435b5af (127.15.81.196:43695): Couldn't send request to peer 4a90097b2af84557940df6589435b5af. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d862a5c73b9242429e1974df9e40e99b. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:15:17.082661 16430 raft_consensus.cc:2955] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } attrs { replace: true } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.083724 16251 catalog_manager.cc:5649] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 reported cstate change: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } attrs { replace: true } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.093626 16430 consensus_queue.cc:237] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } }
I20251023 20:15:17.095813 16504 raft_consensus.cc:1275] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Refusing update from remote peer e36e6090c1534a159466eba7a367b06a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.095814 16354 raft_consensus.cc:1275] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Refusing update from remote peer e36e6090c1534a159466eba7a367b06a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.096899 16711 consensus_queue.cc:1048] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.097240 16556 consensus_queue.cc:1048] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.100025 16556 raft_consensus.cc:2955] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.103298 16504 raft_consensus.cc:2955] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
W20251023 20:15:17.104146 16394 consensus_peers.cc:597] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a -> Peer 4a90097b2af84557940df6589435b5af (127.15.81.196:43695): Couldn't send request to peer 4a90097b2af84557940df6589435b5af. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7b25ea079ba24e5a8f9caa3a868b77fa. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:15:17.107317 16250 catalog_manager.cc:5649] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 reported cstate change: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.108234 16430 consensus_queue.cc:237] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } attrs { replace: true } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } }
I20251023 20:15:17.110203 16355 raft_consensus.cc:1275] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Refusing update from remote peer e36e6090c1534a159466eba7a367b06a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 20:15:17.110316 16394 consensus_peers.cc:597] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a -> Peer 4a90097b2af84557940df6589435b5af (127.15.81.196:43695): Couldn't send request to peer 4a90097b2af84557940df6589435b5af. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8963a2dc524541858760687a81b55cbf. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:15:17.110594 16504 raft_consensus.cc:1275] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Refusing update from remote peer e36e6090c1534a159466eba7a367b06a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.111004 16712 consensus_queue.cc:1048] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.111524 16711 consensus_queue.cc:1048] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.115028 16711 raft_consensus.cc:2955] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } attrs { replace: true } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.115587 16504 raft_consensus.cc:2955] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } attrs { replace: true } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.120635 16430 consensus_queue.cc:237] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } }
I20251023 20:15:17.120348 16354 raft_consensus.cc:2955] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.122706 16249 catalog_manager.cc:5649] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a reported cstate change: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:17.127080 16504 raft_consensus.cc:1275] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Refusing update from remote peer e36e6090c1534a159466eba7a367b06a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 20:15:17.127560 16394 consensus_peers.cc:597] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a -> Peer 4a90097b2af84557940df6589435b5af (127.15.81.196:43695): Couldn't send request to peer 4a90097b2af84557940df6589435b5af. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f00a0736570c4022964eb3d775a116d2. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:15:17.127614 16712 consensus_queue.cc:1048] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.129721 16355 raft_consensus.cc:1275] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Refusing update from remote peer e36e6090c1534a159466eba7a367b06a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.129815 16354 raft_consensus.cc:2955] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } attrs { replace: true } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.130366 16711 raft_consensus.cc:2955] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.131033 16556 consensus_queue.cc:1048] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:17.135457 16504 raft_consensus.cc:2955] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.135610 16250 catalog_manager.cc:5649] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a reported cstate change: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "e36e6090c1534a159466eba7a367b06a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:17.134122 16355 raft_consensus.cc:2955] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4a90097b2af84557940df6589435b5af (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c1b215e0869a4027834da3a6dcfea3e6" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35787 } attrs { replace: true } } peers { permanent_uuid: "e36e6090c1534a159466eba7a367b06a" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 41165 } } peers { permanent_uuid: "3a3ba6bf43894c9fa6e62c2c3932bfd5" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37183 } } peers { permanent_uuid: "4a90097b2af84557940df6589435b5af" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 43695 } attrs { promote: true } } }
I20251023 20:15:17.172317 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:15:17.180996 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:17.181268 15687 tablet_replica.cc:333] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.181531 15687 raft_consensus.cc:2243] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.181716 15687 raft_consensus.cc:2272] T caa0acadb3814c2c9b3237325cba46ac P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.182336 15687 tablet_replica.cc:333] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.182511 15687 raft_consensus.cc:2243] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.182723 15687 raft_consensus.cc:2272] T 8963a2dc524541858760687a81b55cbf P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.183341 15687 tablet_replica.cc:333] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.183476 15687 raft_consensus.cc:2243] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.183678 15687 raft_consensus.cc:2272] T 7b25ea079ba24e5a8f9caa3a868b77fa P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.184337 15687 tablet_replica.cc:333] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.184526 15687 raft_consensus.cc:2243] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.184695 15687 raft_consensus.cc:2272] T f00a0736570c4022964eb3d775a116d2 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.185334 15687 tablet_replica.cc:333] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.185499 15687 raft_consensus.cc:2243] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.185758 15687 raft_consensus.cc:2272] T 9866d8ae70484eaaa2766c24d15ebc76 P c1b215e0869a4027834da3a6dcfea3e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.186187 15687 tablet_replica.cc:333] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.186350 15687 raft_consensus.cc:2243] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.186522 15687 raft_consensus.cc:2272] T d862a5c73b9242429e1974df9e40e99b P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.187163 15687 tablet_replica.cc:333] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.187323 15687 raft_consensus.cc:2243] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.187602 15687 raft_consensus.cc:2272] T cdcad6626a79486d944c3c177b6564bd P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.187991 15687 tablet_replica.cc:333] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6: stopping tablet replica
I20251023 20:15:17.188175 15687 raft_consensus.cc:2243] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.188419 15687 raft_consensus.cc:2272] T 8eb6f20483e14b9fb5d019c70e804459 P c1b215e0869a4027834da3a6dcfea3e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.203325 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:15:17.208039 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
W20251023 20:15:17.209458 16238 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20251023 20:15:17.211073 16300 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:15:17.211361 16300 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:15:17.215148 16300 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:15:17.215227 16300 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 20:15:17.217726 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:17.217969 15687 tablet_replica.cc:333] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.218190 15687 raft_consensus.cc:2243] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.218372 15687 raft_consensus.cc:2272] T caa0acadb3814c2c9b3237325cba46ac P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
W20251023 20:15:17.218873 16300 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:15:17.218950 16300 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 20:15:17.218981 15687 tablet_replica.cc:333] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.219141 15687 raft_consensus.cc:2243] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.219527 15687 raft_consensus.cc:2272] T 7b25ea079ba24e5a8f9caa3a868b77fa P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.220113 15687 tablet_replica.cc:333] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.220283 15687 raft_consensus.cc:2243] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.220422 15687 raft_consensus.cc:2272] T 8eb6f20483e14b9fb5d019c70e804459 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.220923 15687 tablet_replica.cc:333] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.221077 15687 raft_consensus.cc:2243] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.221227 15687 raft_consensus.cc:2272] T 9866d8ae70484eaaa2766c24d15ebc76 P e36e6090c1534a159466eba7a367b06a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.221678 15687 tablet_replica.cc:333] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.221850 15687 raft_consensus.cc:2243] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.222002 15687 raft_consensus.cc:2272] T cdcad6626a79486d944c3c177b6564bd P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.222398 15687 tablet_replica.cc:333] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.222530 15687 raft_consensus.cc:2243] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.222720 15687 raft_consensus.cc:2272] T d862a5c73b9242429e1974df9e40e99b P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.223239 15687 tablet_replica.cc:333] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.223381 15687 raft_consensus.cc:2243] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.223647 15687 raft_consensus.cc:2272] T 8963a2dc524541858760687a81b55cbf P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.224186 15687 tablet_replica.cc:333] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a: stopping tablet replica
I20251023 20:15:17.224318 15687 raft_consensus.cc:2243] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.224612 15687 raft_consensus.cc:2272] T f00a0736570c4022964eb3d775a116d2 P e36e6090c1534a159466eba7a367b06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.240721 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:15:17.245162 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
W20251023 20:15:17.247419 16300 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:15:17.247979 16300 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 20:15:17.253856 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:17.254055 15687 tablet_replica.cc:333] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.254216 15687 raft_consensus.cc:2243] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.254392 15687 raft_consensus.cc:2272] T 9866d8ae70484eaaa2766c24d15ebc76 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.254850 15687 tablet_replica.cc:333] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.254998 15687 raft_consensus.cc:2243] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.255147 15687 raft_consensus.cc:2272] T 7b25ea079ba24e5a8f9caa3a868b77fa P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.255667 15687 tablet_replica.cc:333] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.255802 15687 raft_consensus.cc:2243] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.256057 15687 raft_consensus.cc:2272] T caa0acadb3814c2c9b3237325cba46ac P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.256450 15687 tablet_replica.cc:333] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.256597 15687 raft_consensus.cc:2243] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.256735 15687 raft_consensus.cc:2272] T cdcad6626a79486d944c3c177b6564bd P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.257174 15687 tablet_replica.cc:333] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.257318 15687 raft_consensus.cc:2243] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.257618 15687 raft_consensus.cc:2272] T d862a5c73b9242429e1974df9e40e99b P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.258109 15687 tablet_replica.cc:333] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.258253 15687 raft_consensus.cc:2243] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.258401 15687 raft_consensus.cc:2272] T 8963a2dc524541858760687a81b55cbf P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.259002 15687 tablet_replica.cc:333] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.259126 15687 raft_consensus.cc:2243] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.259261 15687 raft_consensus.cc:2272] T 8eb6f20483e14b9fb5d019c70e804459 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.259617 15687 tablet_replica.cc:333] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5: stopping tablet replica
I20251023 20:15:17.259755 15687 raft_consensus.cc:2243] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:17.259898 15687 raft_consensus.cc:2272] T f00a0736570c4022964eb3d775a116d2 P 3a3ba6bf43894c9fa6e62c2c3932bfd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.264638 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:15:17.268065 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:15:17.275146 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:17.287523 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:15:17.289832 15687 master.cc:561] Master@127.15.81.254:33745 shutting down...
I20251023 20:15:17.296363 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:17.296540 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:17.296629 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 255d9a20a205458c94f05b50910730d8: stopping tablet replica
I20251023 20:15:17.299908 15687 master.cc:583] Master@127.15.81.254:33745 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn (7325 ms)
[ RUN ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing
I20251023 20:15:17.315117 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:45479,127.15.81.253:37925,127.15.81.252:35569
I20251023 20:15:17.315642 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:17.317893 16726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.317972 16727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.318549 16729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.318696 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:17.318967 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:17.319027 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:17.319096 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250517319067 us; error 0 us; skew 500 ppm
I20251023 20:15:17.319252 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:17.320277 15687 webserver.cc:492] Webserver started at http://127.15.81.254:34285/ using document root <none> and password file <none>
I20251023 20:15:17.320472 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:17.320559 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:17.320698 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:17.321372 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "b3a9d9c6d14e4063b663faa6d4daca01"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.323724 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:17.325137 16734 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.325438 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:17.325532 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "b3a9d9c6d14e4063b663faa6d4daca01"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.325685 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:17.351506 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:17.352088 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:17.364789 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:45479
I20251023 20:15:17.364820 16786 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:45479 every 8 connection(s)
I20251023 20:15:17.366012 16787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:17.366086 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:17.368335 16789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.369263 16790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.369740 16787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
W20251023 20:15:17.370440 16792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.370784 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:17.371241 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:17.371327 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:17.371412 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250517371369 us; error 0 us; skew 500 ppm
I20251023 20:15:17.371565 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:17.372789 15687 webserver.cc:492] Webserver started at http://127.15.81.253:37117/ using document root <none> and password file <none>
I20251023 20:15:17.373005 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:17.373138 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:17.373271 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:17.373828 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root/instance:
uuid: "1e5cc310ec4f413d89ac661f956d8781"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.376296 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:17.377364 16787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.378083 16799 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.378448 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:17.378566 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root
uuid: "1e5cc310ec4f413d89ac661f956d8781"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.378700 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251023 20:15:17.379168 16787 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.81.253:37925: Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:17.390630 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:17.391121 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:17.403247 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.253:37925
I20251023 20:15:17.403281 16852 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.253:37925 every 8 connection(s)
I20251023 20:15:17.404563 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 20:15:17.404608 16853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251023 20:15:17.406981 16855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.408067 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.409150 16858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.409088 16853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.414300 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:17.414713 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:17.414834 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:17.414947 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250517414911 us; error 0 us; skew 500 ppm
I20251023 20:15:17.415138 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:17.416450 15687 webserver.cc:492] Webserver started at http://127.15.81.252:41719/ using document root <none> and password file <none>
I20251023 20:15:17.416553 16853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.416724 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:17.416832 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:17.416988 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:17.417807 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root/instance:
uuid: "5e46467377c04e72a1fa9c490f093896"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.420266 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:15:17.422210 16865 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.422607 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:17.422755 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root
uuid: "5e46467377c04e72a1fa9c490f093896"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.422868 16853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.422905 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251023 20:15:17.424809 16853 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.81.252:35569: Network error: Client connection negotiation failed: client connection to 127.15.81.252:35569: connect: Connection refused (error 111)
I20251023 20:15:17.442384 16787 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } attempt: 1
I20251023 20:15:17.447685 16787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
W20251023 20:15:17.449167 16787 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.81.252:35569: Network error: Client connection negotiation failed: client connection to 127.15.81.252:35569: connect: Connection refused (error 111)
I20251023 20:15:17.451016 16853 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } attempt: 1
W20251023 20:15:17.452284 16853 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.81.252:35569: Network error: Client connection negotiation failed: client connection to 127.15.81.252:35569: connect: Connection refused (error 111)
I20251023 20:15:17.463258 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:17.463821 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:17.476485 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.252:35569
I20251023 20:15:17.476522 16918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.252:35569 every 8 connection(s)
I20251023 20:15:17.477393 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:15:17.477860 16919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:17.480039 16919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.480363 16787 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } attempt: 1
I20251023 20:15:17.486836 16919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.489734 16787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: Bootstrap starting.
I20251023 20:15:17.491557 16787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:17.492811 16919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } has no permanent_uuid. Determining permanent_uuid...
I20251023 20:15:17.493624 16787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: No bootstrap required, opened a new log
I20251023 20:15:17.494500 16787 raft_consensus.cc:359] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.494764 16787 raft_consensus.cc:385] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:17.494865 16787 raft_consensus.cc:740] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a9d9c6d14e4063b663faa6d4daca01, State: Initialized, Role: FOLLOWER
I20251023 20:15:17.495114 16787 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.495743 16924 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.495939 16787 sys_catalog.cc:565] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:17.495954 16924 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.500454 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251023 20:15:17.501690 16935 catalog_manager.cc:1568] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:15:17.501796 16935 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 20:15:17.502278 16919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: Bootstrap starting.
I20251023 20:15:17.503476 16919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:17.504933 16919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: No bootstrap required, opened a new log
I20251023 20:15:17.505466 16919 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.505615 16919 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:17.505672 16919 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e46467377c04e72a1fa9c490f093896, State: Initialized, Role: FOLLOWER
I20251023 20:15:17.505882 16919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.506371 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.506547 16919 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:17.506569 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.507495 16853 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } attempt: 2
W20251023 20:15:17.512256 16948 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:15:17.512389 16948 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 20:15:17.515722 16853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781: Bootstrap starting.
I20251023 20:15:17.516983 16853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:17.518440 16853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781: No bootstrap required, opened a new log
I20251023 20:15:17.519001 16853 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.519155 16853 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:17.519230 16853 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5cc310ec4f413d89ac661f956d8781, State: Initialized, Role: FOLLOWER
I20251023 20:15:17.519443 16853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.519960 16950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.520120 16950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.520154 16853 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:17.523536 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251023 20:15:17.524366 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:17.524789 16961 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:15:17.524923 16961 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20251023 20:15:17.527114 16962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.527480 16963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.529698 16965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.530853 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:17.531100 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:17.531178 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:17.531258 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250517531236 us; error 0 us; skew 500 ppm
I20251023 20:15:17.531421 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:17.532473 15687 webserver.cc:492] Webserver started at http://127.15.81.193:39097/ using document root <none> and password file <none>
I20251023 20:15:17.532683 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:17.532780 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:17.532953 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:17.533542 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "735ed38b18594e438346a1c5af4e541a"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.535868 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:17.537504 16970 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.537832 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251023 20:15:17.537935 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "735ed38b18594e438346a1c5af4e541a"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.538049 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:17.551749 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:17.552275 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:17.552810 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:17.553707 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:17.553807 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.553910 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:17.553989 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.569257 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:37665
I20251023 20:15:17.569310 17032 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:37665 every 8 connection(s)
I20251023 20:15:17.575034 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:17.581804 17040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.583250 17035 heartbeater.cc:344] Connected to a master server at 127.15.81.253:37925
I20251023 20:15:17.583395 17035 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.583897 17035 heartbeater.cc:507] Master 127.15.81.253:37925 requested a full tablet report, sending...
I20251023 20:15:17.584458 17034 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45479
I20251023 20:15:17.584621 17034 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.584934 17033 heartbeater.cc:344] Connected to a master server at 127.15.81.252:35569
I20251023 20:15:17.585055 17033 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.585809 17033 heartbeater.cc:507] Master 127.15.81.252:35569 requested a full tablet report, sending...
I20251023 20:15:17.586287 16818 ts_manager.cc:194] Registered new tserver with Master: 735ed38b18594e438346a1c5af4e541a (127.15.81.193:37665)
I20251023 20:15:17.586935 17034 heartbeater.cc:507] Master 127.15.81.254:45479 requested a full tablet report, sending...
W20251023 20:15:17.586666 17041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.587594 16883 ts_manager.cc:194] Registered new tserver with Master: 735ed38b18594e438346a1c5af4e541a (127.15.81.193:37665)
I20251023 20:15:17.588455 16751 ts_manager.cc:194] Registered new tserver with Master: 735ed38b18594e438346a1c5af4e541a (127.15.81.193:37665)
I20251023 20:15:17.589356 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:17.589452 17043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.589794 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:17.589860 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:17.589918 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250517589889 us; error 0 us; skew 500 ppm
I20251023 20:15:17.590034 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:17.591145 15687 webserver.cc:492] Webserver started at http://127.15.81.194:33953/ using document root <none> and password file <none>
I20251023 20:15:17.591339 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:17.591414 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:17.591514 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:17.592056 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "00993762496347f1b1d56b05a069d237"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.594223 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:17.595714 17048 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.596015 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:17.596144 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "00993762496347f1b1d56b05a069d237"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.596266 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:17.638222 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:17.638901 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:17.639483 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:17.640304 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:17.640390 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.640523 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:17.640580 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.656898 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:44865
I20251023 20:15:17.656937 17110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:44865 every 8 connection(s)
I20251023 20:15:17.678453 17112 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45479
I20251023 20:15:17.678526 17113 heartbeater.cc:344] Connected to a master server at 127.15.81.253:37925
I20251023 20:15:17.678726 17112 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.678758 17113 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.678968 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 20:15:17.678978 17111 heartbeater.cc:344] Connected to a master server at 127.15.81.252:35569
I20251023 20:15:17.679225 17111 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.679311 17113 heartbeater.cc:507] Master 127.15.81.253:37925 requested a full tablet report, sending...
I20251023 20:15:17.679375 17112 heartbeater.cc:507] Master 127.15.81.254:45479 requested a full tablet report, sending...
I20251023 20:15:17.679689 17111 heartbeater.cc:507] Master 127.15.81.252:35569 requested a full tablet report, sending...
I20251023 20:15:17.680398 16751 ts_manager.cc:194] Registered new tserver with Master: 00993762496347f1b1d56b05a069d237 (127.15.81.194:44865)
I20251023 20:15:17.680399 16818 ts_manager.cc:194] Registered new tserver with Master: 00993762496347f1b1d56b05a069d237 (127.15.81.194:44865)
I20251023 20:15:17.680629 16883 ts_manager.cc:194] Registered new tserver with Master: 00993762496347f1b1d56b05a069d237 (127.15.81.194:44865)
W20251023 20:15:17.683458 17119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.683532 17118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:17.684022 17121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:17.684480 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:17.684705 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:17.684777 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:17.684820 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250517684807 us; error 0 us; skew 500 ppm
I20251023 20:15:17.684981 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:17.686071 15687 webserver.cc:492] Webserver started at http://127.15.81.195:42529/ using document root <none> and password file <none>
I20251023 20:15:17.686269 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:17.686333 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:17.686443 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:17.686983 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "7c27beaef80048ae991da82ed73490f1"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.689215 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:17.690626 17126 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.690956 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:17.691045 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "7c27beaef80048ae991da82ed73490f1"
format_stamp: "Formatted at 2025-10-23 20:15:17 on dist-test-slave-jr94"
I20251023 20:15:17.691133 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:17.703235 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:17.703691 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:17.704191 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:17.704974 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:17.705052 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.705121 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:17.705173 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:17.722362 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:34613
I20251023 20:15:17.722396 17188 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:34613 every 8 connection(s)
I20251023 20:15:17.738852 17190 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45479
I20251023 20:15:17.739463 17190 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.739691 17189 heartbeater.cc:344] Connected to a master server at 127.15.81.252:35569
I20251023 20:15:17.739853 17189 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.740031 17190 heartbeater.cc:507] Master 127.15.81.254:45479 requested a full tablet report, sending...
I20251023 20:15:17.740326 17189 heartbeater.cc:507] Master 127.15.81.252:35569 requested a full tablet report, sending...
I20251023 20:15:17.741654 16883 ts_manager.cc:194] Registered new tserver with Master: 7c27beaef80048ae991da82ed73490f1 (127.15.81.195:34613)
I20251023 20:15:17.742066 17191 heartbeater.cc:344] Connected to a master server at 127.15.81.253:37925
I20251023 20:15:17.742220 17191 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:17.742637 17191 heartbeater.cc:507] Master 127.15.81.253:37925 requested a full tablet report, sending...
I20251023 20:15:17.743623 16818 ts_manager.cc:194] Registered new tserver with Master: 7c27beaef80048ae991da82ed73490f1 (127.15.81.195:34613)
I20251023 20:15:17.743649 16751 ts_manager.cc:194] Registered new tserver with Master: 7c27beaef80048ae991da82ed73490f1 (127.15.81.195:34613)
I20251023 20:15:17.744431 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01404822s
I20251023 20:15:17.761282 16950 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:17.761528 16950 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.762398 16950 leader_election.cc:290] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3a9d9c6d14e4063b663faa6d4daca01 (127.15.81.254:45479), 5e46467377c04e72a1fa9c490f093896 (127.15.81.252:35569)
I20251023 20:15:17.762866 16761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e5cc310ec4f413d89ac661f956d8781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" is_pre_election: true
I20251023 20:15:17.763093 16761 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e5cc310ec4f413d89ac661f956d8781 in term 0.
I20251023 20:15:17.763451 16893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e5cc310ec4f413d89ac661f956d8781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e46467377c04e72a1fa9c490f093896" is_pre_election: true
I20251023 20:15:17.763549 16803 leader_election.cc:304] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e5cc310ec4f413d89ac661f956d8781, b3a9d9c6d14e4063b663faa6d4daca01; no voters:
I20251023 20:15:17.763686 16893 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e5cc310ec4f413d89ac661f956d8781 in term 0.
I20251023 20:15:17.763866 16950 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:17.763995 16950 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:17.764094 16950 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:17.766773 16950 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.767376 16950 leader_election.cc:290] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [CANDIDATE]: Term 1 election: Requested vote from peers b3a9d9c6d14e4063b663faa6d4daca01 (127.15.81.254:45479), 5e46467377c04e72a1fa9c490f093896 (127.15.81.252:35569)
I20251023 20:15:17.767802 16761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e5cc310ec4f413d89ac661f956d8781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a9d9c6d14e4063b663faa6d4daca01"
I20251023 20:15:17.767823 16893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e5cc310ec4f413d89ac661f956d8781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e46467377c04e72a1fa9c490f093896"
I20251023 20:15:17.768011 16761 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:17.768013 16893 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:17.770853 16893 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e5cc310ec4f413d89ac661f956d8781 in term 1.
I20251023 20:15:17.771160 16761 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e5cc310ec4f413d89ac661f956d8781 in term 1.
I20251023 20:15:17.771219 16802 leader_election.cc:304] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e5cc310ec4f413d89ac661f956d8781, 5e46467377c04e72a1fa9c490f093896; no voters:
I20251023 20:15:17.771524 16950 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:17.771996 16950 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 1 LEADER]: Becoming Leader. State: Replica: 1e5cc310ec4f413d89ac661f956d8781, State: Running, Role: LEADER
I20251023 20:15:17.772330 16950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:17.773986 17197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e5cc310ec4f413d89ac661f956d8781. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.774335 17197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:17.775023 17199 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:15:17.776353 17199 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:15:17.779801 16893 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Refusing update from remote peer 1e5cc310ec4f413d89ac661f956d8781: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.779912 16761 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Refusing update from remote peer 1e5cc310ec4f413d89ac661f956d8781: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:17.780344 16950 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:17.780531 17197 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:17.785430 16924 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e5cc310ec4f413d89ac661f956d8781. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.785673 16924 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.786626 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e5cc310ec4f413d89ac661f956d8781. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.786857 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.787762 17197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.787988 17197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:17.788182 17197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.788357 17197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:17.788553 17199 catalog_manager.cc:1357] Generated new cluster ID: aceead8ab2ed4fdc94c6bd6655debacb
I20251023 20:15:17.788648 17199 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:15:17.789031 16937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.789273 16937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.790597 16924 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e5cc310ec4f413d89ac661f956d8781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:17.790797 16924 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 20:15:17.791431 17200 mvcc.cc:204] Tried to move back new op lower bound from 7214082120821088256 to 7214082120798715904. Current Snapshot: MvccSnapshot[applied={T|T < 7214082120821088256 or (T in {7214082120821088256})}]
I20251023 20:15:17.798671 17199 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:15:17.799714 17199 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:15:17.811100 17199 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781: Generated new TSK 0
I20251023 20:15:17.811348 17199 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:15:18.503013 16935 catalog_manager.cc:1269] Loaded cluster ID: aceead8ab2ed4fdc94c6bd6655debacb
I20251023 20:15:18.503122 16935 catalog_manager.cc:1562] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: loading cluster ID for follower catalog manager: success
I20251023 20:15:18.504768 16935 catalog_manager.cc:1584] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: acquiring CA information for follower catalog manager: success
I20251023 20:15:18.505648 16935 catalog_manager.cc:1612] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 20:15:18.513435 16948 catalog_manager.cc:1269] Loaded cluster ID: aceead8ab2ed4fdc94c6bd6655debacb
I20251023 20:15:18.513548 16948 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: loading cluster ID for follower catalog manager: success
I20251023 20:15:18.515152 16948 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: acquiring CA information for follower catalog manager: success
I20251023 20:15:18.515975 16948 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 20:15:18.589133 16818 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37764
I20251023 20:15:18.683264 16818 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37774
I20251023 20:15:18.745625 16818 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37776
I20251023 20:15:19.523492 16960 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:19.590541 17035 heartbeater.cc:499] Master 127.15.81.253:37925 was elected leader, sending a full tablet report...
I20251023 20:15:19.684309 17113 heartbeater.cc:499] Master 127.15.81.253:37925 was elected leader, sending a full tablet report...
I20251023 20:15:19.746659 17191 heartbeater.cc:499] Master 127.15.81.253:37925 was elected leader, sending a full tablet report...
I20251023 20:15:19.903062 15687 test_util.cc:276] Using random seed: 395930433
I20251023 20:15:19.929453 16818 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37782:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251023 20:15:19.987867 17076 tablet_service.cc:1505] Processing CreateTablet for tablet e95a9feddef344baac85a3c6d78ad8c8 (DEFAULT_TABLE table=test-workload [id=3c559320a4ce41c38b6d11e30337912b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 20:15:19.988404 17076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95a9feddef344baac85a3c6d78ad8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:19.992679 16998 tablet_service.cc:1505] Processing CreateTablet for tablet 8abf982d62b442f69ae7e7bbfccd9476 (DEFAULT_TABLE table=test-workload [id=3c559320a4ce41c38b6d11e30337912b]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 20:15:19.993122 16998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8abf982d62b442f69ae7e7bbfccd9476. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:19.993865 17236 tablet_bootstrap.cc:492] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237: Bootstrap starting.
I20251023 20:15:19.995595 17236 tablet_bootstrap.cc:654] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:19.997983 17154 tablet_service.cc:1505] Processing CreateTablet for tablet f9f79f69141944f292bc53c550ef7fec (DEFAULT_TABLE table=test-workload [id=3c559320a4ce41c38b6d11e30337912b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 20:15:19.998175 17236 tablet_bootstrap.cc:492] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237: No bootstrap required, opened a new log
I20251023 20:15:19.998368 17236 ts_tablet_manager.cc:1403] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251023 20:15:19.998446 17154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f79f69141944f292bc53c550ef7fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:19.999378 17236 raft_consensus.cc:359] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00993762496347f1b1d56b05a069d237" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 44865 } }
I20251023 20:15:19.999564 17236 raft_consensus.cc:385] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:19.999666 17236 raft_consensus.cc:740] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00993762496347f1b1d56b05a069d237, State: Initialized, Role: FOLLOWER
I20251023 20:15:19.999761 17238 tablet_bootstrap.cc:492] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a: Bootstrap starting.
I20251023 20:15:19.999966 17236 consensus_queue.cc:260] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00993762496347f1b1d56b05a069d237" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 44865 } }
I20251023 20:15:20.000173 17236 raft_consensus.cc:399] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:20.000262 17236 raft_consensus.cc:493] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:20.000389 17236 raft_consensus.cc:3060] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:20.001492 17238 tablet_bootstrap.cc:654] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:20.003273 17236 raft_consensus.cc:515] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00993762496347f1b1d56b05a069d237" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 44865 } }
I20251023 20:15:20.003477 17238 tablet_bootstrap.cc:492] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a: No bootstrap required, opened a new log
I20251023 20:15:20.003535 17236 leader_election.cc:304] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00993762496347f1b1d56b05a069d237; no voters:
I20251023 20:15:20.003620 17238 ts_tablet_manager.cc:1403] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:15:20.004082 17236 leader_election.cc:290] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:20.004266 17240 raft_consensus.cc:2804] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:20.004702 17238 raft_consensus.cc:359] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ed38b18594e438346a1c5af4e541a" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37665 } }
I20251023 20:15:20.005002 17238 raft_consensus.cc:385] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:20.005178 17238 raft_consensus.cc:740] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735ed38b18594e438346a1c5af4e541a, State: Initialized, Role: FOLLOWER
I20251023 20:15:20.005550 17238 consensus_queue.cc:260] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ed38b18594e438346a1c5af4e541a" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37665 } }
I20251023 20:15:20.005848 17238 raft_consensus.cc:399] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:20.005991 17238 raft_consensus.cc:493] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:20.006166 17238 raft_consensus.cc:3060] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:20.009454 17238 raft_consensus.cc:515] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ed38b18594e438346a1c5af4e541a" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37665 } }
I20251023 20:15:20.009943 17238 leader_election.cc:304] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735ed38b18594e438346a1c5af4e541a; no voters:
I20251023 20:15:20.010110 17243 tablet_bootstrap.cc:492] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1: Bootstrap starting.
I20251023 20:15:20.011463 17240 raft_consensus.cc:697] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 1 LEADER]: Becoming Leader. State: Replica: 00993762496347f1b1d56b05a069d237, State: Running, Role: LEADER
I20251023 20:15:20.011911 17240 consensus_queue.cc:237] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00993762496347f1b1d56b05a069d237" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 44865 } }
I20251023 20:15:20.013309 17243 tablet_bootstrap.cc:654] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:20.013957 17236 ts_tablet_manager.cc:1434] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237: Time spent starting tablet: real 0.015s user 0.003s sys 0.012s
I20251023 20:15:20.014559 17244 raft_consensus.cc:2804] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:20.014606 17238 leader_election.cc:290] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:20.017328 16818 catalog_manager.cc:5649] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00993762496347f1b1d56b05a069d237 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "00993762496347f1b1d56b05a069d237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00993762496347f1b1d56b05a069d237" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 44865 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:20.023540 17243 tablet_bootstrap.cc:492] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1: No bootstrap required, opened a new log
I20251023 20:15:20.023802 17243 ts_tablet_manager.cc:1403] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20251023 20:15:20.024885 17243 raft_consensus.cc:359] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c27beaef80048ae991da82ed73490f1" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34613 } }
I20251023 20:15:20.025149 17243 raft_consensus.cc:385] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:20.025322 17243 raft_consensus.cc:740] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c27beaef80048ae991da82ed73490f1, State: Initialized, Role: FOLLOWER
I20251023 20:15:20.025174 17238 ts_tablet_manager.cc:1434] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a: Time spent starting tablet: real 0.021s user 0.015s sys 0.000s
I20251023 20:15:20.026820 17243 consensus_queue.cc:260] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c27beaef80048ae991da82ed73490f1" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34613 } }
I20251023 20:15:20.027146 17243 raft_consensus.cc:399] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:20.027293 17243 raft_consensus.cc:493] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:20.027484 17243 raft_consensus.cc:3060] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:20.027868 17244 raft_consensus.cc:697] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 1 LEADER]: Becoming Leader. State: Replica: 735ed38b18594e438346a1c5af4e541a, State: Running, Role: LEADER
I20251023 20:15:20.028219 17244 consensus_queue.cc:237] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ed38b18594e438346a1c5af4e541a" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37665 } }
I20251023 20:15:20.030771 17243 raft_consensus.cc:515] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c27beaef80048ae991da82ed73490f1" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34613 } }
I20251023 20:15:20.031085 17243 leader_election.cc:304] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c27beaef80048ae991da82ed73490f1; no voters:
I20251023 20:15:20.035626 16817 catalog_manager.cc:5649] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a reported cstate change: term changed from 0 to 1, leader changed from <none> to 735ed38b18594e438346a1c5af4e541a (127.15.81.193). New cstate: current_term: 1 leader_uuid: "735ed38b18594e438346a1c5af4e541a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ed38b18594e438346a1c5af4e541a" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37665 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:20.045164 17243 leader_election.cc:290] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:20.045347 17251 raft_consensus.cc:2804] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:20.049521 17243 ts_tablet_manager.cc:1434] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1: Time spent starting tablet: real 0.026s user 0.008s sys 0.015s
I20251023 20:15:20.049607 17251 raft_consensus.cc:697] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c27beaef80048ae991da82ed73490f1, State: Running, Role: LEADER
I20251023 20:15:20.049913 17251 consensus_queue.cc:237] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c27beaef80048ae991da82ed73490f1" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34613 } }
I20251023 20:15:20.052170 16817 catalog_manager.cc:5649] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c27beaef80048ae991da82ed73490f1 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "7c27beaef80048ae991da82ed73490f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c27beaef80048ae991da82ed73490f1" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34613 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:20.065845 15687 master.cc:561] Master@127.15.81.253:37925 shutting down...
I20251023 20:15:20.071534 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:20.071857 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:20.071990 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1e5cc310ec4f413d89ac661f956d8781: stopping tablet replica
I20251023 20:15:20.086369 15687 master.cc:583] Master@127.15.81.253:37925 shutdown complete.
W20251023 20:15:21.037617 17113 heartbeater.cc:646] Failed to heartbeat to 127.15.81.253:37925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:21.557531 17261 raft_consensus.cc:493] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1e5cc310ec4f413d89ac661f956d8781)
I20251023 20:15:21.557631 17262 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1e5cc310ec4f413d89ac661f956d8781)
I20251023 20:15:21.557821 17262 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:21.557822 17261 raft_consensus.cc:515] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:21.558769 17261 leader_election.cc:290] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925), 5e46467377c04e72a1fa9c490f093896 (127.15.81.252:35569)
I20251023 20:15:21.559212 17262 leader_election.cc:290] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3a9d9c6d14e4063b663faa6d4daca01 (127.15.81.254:45479), 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925)
I20251023 20:15:21.559536 16761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e46467377c04e72a1fa9c490f093896" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" is_pre_election: true
I20251023 20:15:21.559808 16761 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e46467377c04e72a1fa9c490f093896 in term 1.
I20251023 20:15:21.559557 16893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "5e46467377c04e72a1fa9c490f093896" is_pre_election: true
I20251023 20:15:21.560513 16868 leader_election.cc:304] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e46467377c04e72a1fa9c490f093896, b3a9d9c6d14e4063b663faa6d4daca01; no voters:
I20251023 20:15:21.560902 16893 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a9d9c6d14e4063b663faa6d4daca01 in term 1.
I20251023 20:15:21.561205 17262 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:15:21.561487 16735 leader_election.cc:304] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e46467377c04e72a1fa9c490f093896, b3a9d9c6d14e4063b663faa6d4daca01; no voters:
I20251023 20:15:21.561487 17262 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1e5cc310ec4f413d89ac661f956d8781)
I20251023 20:15:21.561843 17261 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20251023 20:15:21.561929 16737 leader_election.cc:336] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925): Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:21.561996 17262 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:21.562027 17261 raft_consensus.cc:493] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1e5cc310ec4f413d89ac661f956d8781)
I20251023 20:15:21.562155 17261 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 1 FOLLOWER]: Advancing to term 2
W20251023 20:15:21.562785 16869 leader_election.cc:336] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925): Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:21.564229 17261 raft_consensus.cc:515] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:21.564211 17262 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:21.564771 17262 leader_election.cc:290] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 2 election: Requested vote from peers b3a9d9c6d14e4063b663faa6d4daca01 (127.15.81.254:45479), 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925)
I20251023 20:15:21.565094 17261 leader_election.cc:290] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [CANDIDATE]: Term 2 election: Requested vote from peers 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925), 5e46467377c04e72a1fa9c490f093896 (127.15.81.252:35569)
I20251023 20:15:21.565218 16893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "5e46467377c04e72a1fa9c490f093896"
I20251023 20:15:21.565189 16761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e46467377c04e72a1fa9c490f093896" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b3a9d9c6d14e4063b663faa6d4daca01"
I20251023 20:15:21.565501 16761 raft_consensus.cc:2393] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e46467377c04e72a1fa9c490f093896 in current term 2: Already voted for candidate b3a9d9c6d14e4063b663faa6d4daca01 in this term.
I20251023 20:15:21.565503 16893 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b3a9d9c6d14e4063b663faa6d4daca01 in current term 2: Already voted for candidate 5e46467377c04e72a1fa9c490f093896 in this term.
W20251023 20:15:21.566156 16737 leader_election.cc:336] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925): Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:21.566265 16737 leader_election.cc:304] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3a9d9c6d14e4063b663faa6d4daca01; no voters: 1e5cc310ec4f413d89ac661f956d8781, 5e46467377c04e72a1fa9c490f093896
I20251023 20:15:21.566665 17261 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20251023 20:15:21.566697 16869 leader_election.cc:336] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925): Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:21.566828 16869 leader_election.cc:304] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e46467377c04e72a1fa9c490f093896; no voters: 1e5cc310ec4f413d89ac661f956d8781, b3a9d9c6d14e4063b663faa6d4daca01
I20251023 20:15:21.567071 17262 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251023 20:15:23.255034 17273 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:23.255260 17273 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:23.255865 17273 leader_election.cc:290] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b3a9d9c6d14e4063b663faa6d4daca01 (127.15.81.254:45479), 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925)
I20251023 20:15:23.256373 16761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e46467377c04e72a1fa9c490f093896" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" is_pre_election: true
I20251023 20:15:23.256541 16761 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e46467377c04e72a1fa9c490f093896 in term 2.
W20251023 20:15:23.256799 16869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111) [suppressed 11 similar messages]
I20251023 20:15:23.257012 16868 leader_election.cc:304] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e46467377c04e72a1fa9c490f093896, b3a9d9c6d14e4063b663faa6d4daca01; no voters:
I20251023 20:15:23.257341 17273 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 20:15:23.257478 17273 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:23.257591 17273 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 2 FOLLOWER]: Advancing to term 3
W20251023 20:15:23.258104 16869 leader_election.cc:336] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925): Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:23.259361 17273 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:23.259784 17273 leader_election.cc:290] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 3 election: Requested vote from peers b3a9d9c6d14e4063b663faa6d4daca01 (127.15.81.254:45479), 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925)
I20251023 20:15:23.260102 16761 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e46467377c04e72a1fa9c490f093896" candidate_term: 3 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: false dest_uuid: "b3a9d9c6d14e4063b663faa6d4daca01"
I20251023 20:15:23.260236 16761 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 2 FOLLOWER]: Advancing to term 3
W20251023 20:15:23.260924 16869 leader_election.cc:336] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1e5cc310ec4f413d89ac661f956d8781 (127.15.81.253:37925): Network error: Client connection negotiation failed: client connection to 127.15.81.253:37925: connect: Connection refused (error 111)
I20251023 20:15:23.262224 16761 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e46467377c04e72a1fa9c490f093896 in term 3.
I20251023 20:15:23.262594 16868 leader_election.cc:304] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e46467377c04e72a1fa9c490f093896, b3a9d9c6d14e4063b663faa6d4daca01; no voters: 1e5cc310ec4f413d89ac661f956d8781
I20251023 20:15:23.262846 17273 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:15:23.263170 17273 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 3 LEADER]: Becoming Leader. State: Replica: 5e46467377c04e72a1fa9c490f093896, State: Running, Role: LEADER
I20251023 20:15:23.263392 17273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } }
I20251023 20:15:23.264441 17276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e46467377c04e72a1fa9c490f093896. Latest consensus state: current_term: 3 leader_uuid: "5e46467377c04e72a1fa9c490f093896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a9d9c6d14e4063b663faa6d4daca01" member_type: VOTER last_known_addr { host: "127.15.81.254" port: 45479 } } peers { permanent_uuid: "1e5cc310ec4f413d89ac661f956d8781" member_type: VOTER last_known_addr { host: "127.15.81.253" port: 37925 } } peers { permanent_uuid: "5e46467377c04e72a1fa9c490f093896" member_type: VOTER last_known_addr { host: "127.15.81.252" port: 35569 } } }
I20251023 20:15:23.264679 17276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:23.265223 17278 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:15:23.267383 17278 catalog_manager.cc:679] Loaded metadata for table test-workload [id=3c559320a4ce41c38b6d11e30337912b]
I20251023 20:15:23.268656 17278 tablet_loader.cc:96] loaded metadata for tablet 8abf982d62b442f69ae7e7bbfccd9476 (table test-workload [id=3c559320a4ce41c38b6d11e30337912b])
I20251023 20:15:23.268972 17278 tablet_loader.cc:96] loaded metadata for tablet e95a9feddef344baac85a3c6d78ad8c8 (table test-workload [id=3c559320a4ce41c38b6d11e30337912b])
I20251023 20:15:23.269218 17278 tablet_loader.cc:96] loaded metadata for tablet f9f79f69141944f292bc53c550ef7fec (table test-workload [id=3c559320a4ce41c38b6d11e30337912b])
I20251023 20:15:23.269387 17278 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:15:23.270068 17278 catalog_manager.cc:1269] Loaded cluster ID: aceead8ab2ed4fdc94c6bd6655debacb
I20251023 20:15:23.270135 17278 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:15:23.271116 17278 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:15:23.271809 17278 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: Loaded TSK: 0
I20251023 20:15:23.272336 17278 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:15:23.511533 16947 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:23.511647 16947 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:23.565658 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:15:23.574893 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:23.575196 15687 tablet_replica.cc:333] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a: stopping tablet replica
I20251023 20:15:23.575389 15687 raft_consensus.cc:2243] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:23.575578 15687 raft_consensus.cc:2272] T 8abf982d62b442f69ae7e7bbfccd9476 P 735ed38b18594e438346a1c5af4e541a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:23.589042 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:15:23.591671 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:15:23.600157 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:23.600443 15687 tablet_replica.cc:333] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237: stopping tablet replica
I20251023 20:15:23.600643 15687 raft_consensus.cc:2243] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:23.600898 15687 raft_consensus.cc:2272] T e95a9feddef344baac85a3c6d78ad8c8 P 00993762496347f1b1d56b05a069d237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:23.614217 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:15:23.616773 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:15:23.625377 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:23.625626 15687 tablet_replica.cc:333] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1: stopping tablet replica
I20251023 20:15:23.625824 15687 raft_consensus.cc:2243] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:23.626044 15687 raft_consensus.cc:2272] T f9f79f69141944f292bc53c550ef7fec P 7c27beaef80048ae991da82ed73490f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:23.639269 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:15:23.641945 15687 master.cc:561] Master@127.15.81.254:45479 shutting down...
I20251023 20:15:23.649230 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:23.649446 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:23.649591 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P b3a9d9c6d14e4063b663faa6d4daca01: stopping tablet replica
I20251023 20:15:23.662998 15687 master.cc:583] Master@127.15.81.254:45479 shutdown complete.
I20251023 20:15:23.666381 15687 master.cc:561] Master@127.15.81.252:35569 shutting down...
I20251023 20:15:23.673110 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 3 LEADER]: Raft consensus shutting down.
I20251023 20:15:23.673362 15687 pending_rounds.cc:70] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: Trying to abort 1 pending ops.
I20251023 20:15:23.673529 15687 pending_rounds.cc:77] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: Aborting op as it isn't in flight: id { term: 3 index: 11 } timestamp: 7214082143288848384 op_type: NO_OP noop_request { }
I20251023 20:15:23.673671 15687 raft_consensus.cc:2889] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:15:23.673760 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:23.673900 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5e46467377c04e72a1fa9c490f093896: stopping tablet replica
I20251023 20:15:23.687572 15687 master.cc:583] Master@127.15.81.252:35569 shutdown complete.
[ OK ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing (6383 ms)
[ RUN ] AutoRebalancerTest.MovesScheduledIfAddTserver
I20251023 20:15:23.699132 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:40855
I20251023 20:15:23.699649 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:23.701936 17279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:23.702461 17280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:23.703212 17282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.703943 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:23.704192 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:23.704283 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:23.704339 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250523704321 us; error 0 us; skew 500 ppm
I20251023 20:15:23.704506 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:23.705614 15687 webserver.cc:492] Webserver started at http://127.15.81.254:41525/ using document root <none> and password file <none>
I20251023 20:15:23.705833 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:23.705912 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:23.706060 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:23.706688 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "0a426e1d837b4699b4ca4841d58cbd4b"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.709096 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20251023 20:15:23.710698 17287 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.711031 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 20:15:23.711143 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "0a426e1d837b4699b4ca4841d58cbd4b"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.711251 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:23.734750 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:23.735329 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:23.748404 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:40855
I20251023 20:15:23.748425 17338 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:40855 every 8 connection(s)
I20251023 20:15:23.749668 17339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:23.753484 17339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b: Bootstrap starting.
I20251023 20:15:23.754704 17339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:23.756089 17339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b: No bootstrap required, opened a new log
I20251023 20:15:23.756639 17339 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" member_type: VOTER }
I20251023 20:15:23.756796 17339 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:23.756906 17339 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a426e1d837b4699b4ca4841d58cbd4b, State: Initialized, Role: FOLLOWER
I20251023 20:15:23.757091 17339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" member_type: VOTER }
I20251023 20:15:23.757210 17339 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:23.757263 17339 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:23.757339 17339 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:23.759040 17339 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" member_type: VOTER }
I20251023 20:15:23.759265 17339 leader_election.cc:304] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a426e1d837b4699b4ca4841d58cbd4b; no voters:
I20251023 20:15:23.759951 17339 leader_election.cc:290] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:23.760088 17342 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:23.760730 17342 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 1 LEADER]: Becoming Leader. State: Replica: 0a426e1d837b4699b4ca4841d58cbd4b, State: Running, Role: LEADER
I20251023 20:15:23.761040 17342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" member_type: VOTER }
I20251023 20:15:23.761770 17342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a426e1d837b4699b4ca4841d58cbd4b. Latest consensus state: current_term: 1 leader_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" member_type: VOTER } }
I20251023 20:15:23.761878 17344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a426e1d837b4699b4ca4841d58cbd4b" member_type: VOTER } }
I20251023 20:15:23.762023 17344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:23.762027 17342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:23.764139 17339 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:23.766124 17346 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:15:23.768446 17346 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:15:23.769418 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:15:23.771669 17346 catalog_manager.cc:1357] Generated new cluster ID: 94811f634ac244d99683ec04bcea265c
I20251023 20:15:23.771739 17346 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:15:23.780050 17346 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:15:23.780660 17346 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:15:23.798040 17346 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b: Generated new TSK 0
I20251023 20:15:23.798249 17346 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:15:23.801959 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:23.804616 17360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:23.805050 17361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.806079 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:23.806082 17363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.806571 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:23.806648 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:23.806749 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250523806714 us; error 0 us; skew 500 ppm
I20251023 20:15:23.806927 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:23.808032 15687 webserver.cc:492] Webserver started at http://127.15.81.193:44469/ using document root <none> and password file <none>
I20251023 20:15:23.808261 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:23.808369 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:23.808503 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:23.809172 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "227661b68e6b438ab50331b3ec09fe54"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.811548 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 20:15:23.813102 17368 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.813381 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 20:15:23.813496 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "227661b68e6b438ab50331b3ec09fe54"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.813617 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:23.831530 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:23.832100 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:23.832670 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:23.833545 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:23.833616 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.833743 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:23.833806 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.847654 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:42333
I20251023 20:15:23.847687 17431 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:42333 every 8 connection(s)
I20251023 20:15:23.849323 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:23.852255 17436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:23.852712 17437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.858047 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:23.858088 17440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.858409 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:23.858512 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:23.858587 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250523858549 us; error 0 us; skew 500 ppm
I20251023 20:15:23.858754 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:23.859776 15687 webserver.cc:492] Webserver started at http://127.15.81.194:38859/ using document root <none> and password file <none>
I20251023 20:15:23.859985 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:23.860077 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:23.860210 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:23.860774 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "3db910924ef94d0cb36917e6480483e4"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.862986 17432 heartbeater.cc:344] Connected to a master server at 127.15.81.254:40855
I20251023 20:15:23.863160 17432 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:23.863624 17432 heartbeater.cc:507] Master 127.15.81.254:40855 requested a full tablet report, sending...
I20251023 20:15:23.863605 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 20:15:23.864506 17304 ts_manager.cc:194] Registered new tserver with Master: 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333)
I20251023 20:15:23.865295 17444 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.865736 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:23.865826 17304 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53590
I20251023 20:15:23.865871 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "3db910924ef94d0cb36917e6480483e4"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.866012 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:23.873466 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:23.873955 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:23.874459 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:23.875222 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:23.875303 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.875389 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:23.875465 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.889114 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:39873
I20251023 20:15:23.889122 17508 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:39873 every 8 connection(s)
I20251023 20:15:23.891094 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:23.899905 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.899905 17509 heartbeater.cc:344] Connected to a master server at 127.15.81.254:40855
I20251023 20:15:23.900490 17509 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:23.901008 17509 heartbeater.cc:507] Master 127.15.81.254:40855 requested a full tablet report, sending...
I20251023 20:15:23.902109 17304 ts_manager.cc:194] Registered new tserver with Master: 3db910924ef94d0cb36917e6480483e4 (127.15.81.194:39873)
I20251023 20:15:23.902205 15687 server_base.cc:1047] running on GCE node
W20251023 20:15:23.902424 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.903205 17304 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53596
W20251023 20:15:23.903388 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:23.903862 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:23.903944 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:23.904021 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250523903986 us; error 0 us; skew 500 ppm
I20251023 20:15:23.904218 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:23.905349 15687 webserver.cc:492] Webserver started at http://127.15.81.195:36011/ using document root <none> and password file <none>
I20251023 20:15:23.905572 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:23.905680 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:23.905809 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:23.906388 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "6fa99129dc754d238dcd9cb7ac9d2b57"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.908667 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:23.910060 17520 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.910359 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:23.910465 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "6fa99129dc754d238dcd9cb7ac9d2b57"
format_stamp: "Formatted at 2025-10-23 20:15:23 on dist-test-slave-jr94"
I20251023 20:15:23.910606 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:23.919780 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:23.920269 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:23.920753 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:23.921540 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:23.921612 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.921741 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:23.921801 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:23.934930 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:43645
I20251023 20:15:23.934968 17582 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:43645 every 8 connection(s)
I20251023 20:15:23.941675 17583 heartbeater.cc:344] Connected to a master server at 127.15.81.254:40855
I20251023 20:15:23.941823 17583 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:23.942144 17583 heartbeater.cc:507] Master 127.15.81.254:40855 requested a full tablet report, sending...
I20251023 20:15:23.942920 17304 ts_manager.cc:194] Registered new tserver with Master: 6fa99129dc754d238dcd9cb7ac9d2b57 (127.15.81.195:43645)
I20251023 20:15:23.943527 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007824219s
I20251023 20:15:23.943568 17304 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53610
I20251023 20:15:24.867105 17432 heartbeater.cc:499] Master 127.15.81.254:40855 was elected leader, sending a full tablet report...
I20251023 20:15:24.904593 17509 heartbeater.cc:499] Master 127.15.81.254:40855 was elected leader, sending a full tablet report...
I20251023 20:15:24.945055 17583 heartbeater.cc:499] Master 127.15.81.254:40855 was elected leader, sending a full tablet report...
I20251023 20:15:24.971024 15687 test_util.cc:276] Using random seed: 400998409
I20251023 20:15:24.986831 17304 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55140:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:15:24.987504 17304 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:15:25.004796 17473 tablet_service.cc:1505] Processing CreateTablet for tablet 9f70fefdea4842d0b538316d35440d66 (DEFAULT_TABLE table=test-workload [id=26cb338297e54356aa0eb4105730b808]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251023 20:15:25.004915 17472 tablet_service.cc:1505] Processing CreateTablet for tablet 819a8d5d8dec402d8d750cc1778f93d3 (DEFAULT_TABLE table=test-workload [id=26cb338297e54356aa0eb4105730b808]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251023 20:15:25.005430 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f70fefdea4842d0b538316d35440d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:25.005682 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819a8d5d8dec402d8d750cc1778f93d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:25.018724 17603 tablet_bootstrap.cc:492] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4: Bootstrap starting.
I20251023 20:15:25.020882 17603 tablet_bootstrap.cc:654] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:25.022918 17397 tablet_service.cc:1505] Processing CreateTablet for tablet 9f70fefdea4842d0b538316d35440d66 (DEFAULT_TABLE table=test-workload [id=26cb338297e54356aa0eb4105730b808]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251023 20:15:25.023080 17603 tablet_bootstrap.cc:492] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4: No bootstrap required, opened a new log
I20251023 20:15:25.023000 17396 tablet_service.cc:1505] Processing CreateTablet for tablet 819a8d5d8dec402d8d750cc1778f93d3 (DEFAULT_TABLE table=test-workload [id=26cb338297e54356aa0eb4105730b808]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251023 20:15:25.023182 17603 ts_tablet_manager.cc:1403] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:15:25.023337 17397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f70fefdea4842d0b538316d35440d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:25.023509 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819a8d5d8dec402d8d750cc1778f93d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:25.023893 17603 raft_consensus.cc:359] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.024044 17603 raft_consensus.cc:385] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:25.024091 17603 raft_consensus.cc:740] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db910924ef94d0cb36917e6480483e4, State: Initialized, Role: FOLLOWER
I20251023 20:15:25.024268 17603 consensus_queue.cc:260] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.028256 17547 tablet_service.cc:1505] Processing CreateTablet for tablet 819a8d5d8dec402d8d750cc1778f93d3 (DEFAULT_TABLE table=test-workload [id=26cb338297e54356aa0eb4105730b808]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251023 20:15:25.028785 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819a8d5d8dec402d8d750cc1778f93d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:25.030182 17606 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: Bootstrap starting.
I20251023 20:15:25.031728 17606 tablet_bootstrap.cc:654] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:25.028323 17548 tablet_service.cc:1505] Processing CreateTablet for tablet 9f70fefdea4842d0b538316d35440d66 (DEFAULT_TABLE table=test-workload [id=26cb338297e54356aa0eb4105730b808]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251023 20:15:25.032893 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f70fefdea4842d0b538316d35440d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:25.036659 17608 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: Bootstrap starting.
I20251023 20:15:25.038929 17608 tablet_bootstrap.cc:654] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:25.040419 17606 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: No bootstrap required, opened a new log
I20251023 20:15:25.040584 17606 ts_tablet_manager.cc:1403] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20251023 20:15:25.040966 17608 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: No bootstrap required, opened a new log
I20251023 20:15:25.041095 17608 ts_tablet_manager.cc:1403] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20251023 20:15:25.041301 17606 raft_consensus.cc:359] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.041496 17606 raft_consensus.cc:385] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:25.041563 17606 raft_consensus.cc:740] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227661b68e6b438ab50331b3ec09fe54, State: Initialized, Role: FOLLOWER
I20251023 20:15:25.041762 17608 raft_consensus.cc:359] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.041791 17606 consensus_queue.cc:260] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.041924 17608 raft_consensus.cc:385] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:25.042037 17608 raft_consensus.cc:740] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fa99129dc754d238dcd9cb7ac9d2b57, State: Initialized, Role: FOLLOWER
I20251023 20:15:25.042269 17608 consensus_queue.cc:260] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.044237 17603 ts_tablet_manager.cc:1434] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4: Time spent starting tablet: real 0.021s user 0.013s sys 0.008s
I20251023 20:15:25.044410 17606 ts_tablet_manager.cc:1434] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251023 20:15:25.044785 17606 tablet_bootstrap.cc:492] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54: Bootstrap starting.
I20251023 20:15:25.044894 17608 ts_tablet_manager.cc:1434] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20251023 20:15:25.045154 17608 tablet_bootstrap.cc:492] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: Bootstrap starting.
I20251023 20:15:25.046522 17606 tablet_bootstrap.cc:654] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:25.046729 17603 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4: Bootstrap starting.
I20251023 20:15:25.046870 17608 tablet_bootstrap.cc:654] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:25.048508 17603 tablet_bootstrap.cc:654] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:25.054466 17608 tablet_bootstrap.cc:492] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: No bootstrap required, opened a new log
I20251023 20:15:25.054649 17608 ts_tablet_manager.cc:1403] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20251023 20:15:25.055160 17603 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4: No bootstrap required, opened a new log
I20251023 20:15:25.055321 17603 ts_tablet_manager.cc:1403] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20251023 20:15:25.055354 17608 raft_consensus.cc:359] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.055593 17608 raft_consensus.cc:385] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:25.055689 17608 raft_consensus.cc:740] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fa99129dc754d238dcd9cb7ac9d2b57, State: Initialized, Role: FOLLOWER
I20251023 20:15:25.056013 17603 raft_consensus.cc:359] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.055979 17608 consensus_queue.cc:260] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.056201 17603 raft_consensus.cc:385] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:25.056372 17603 raft_consensus.cc:740] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db910924ef94d0cb36917e6480483e4, State: Initialized, Role: FOLLOWER
I20251023 20:15:25.056615 17608 ts_tablet_manager.cc:1434] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:15:25.056644 17603 consensus_queue.cc:260] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.057314 17606 tablet_bootstrap.cc:492] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54: No bootstrap required, opened a new log
I20251023 20:15:25.057487 17606 ts_tablet_manager.cc:1403] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.009s
I20251023 20:15:25.057559 17603 ts_tablet_manager.cc:1434] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:15:25.058148 17606 raft_consensus.cc:359] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.058334 17606 raft_consensus.cc:385] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:25.058435 17606 raft_consensus.cc:740] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227661b68e6b438ab50331b3ec09fe54, State: Initialized, Role: FOLLOWER
I20251023 20:15:25.058657 17606 consensus_queue.cc:260] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.059048 17606 ts_tablet_manager.cc:1434] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:25.225406 17611 raft_consensus.cc:493] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:25.225577 17611 raft_consensus.cc:515] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.226320 17611 leader_election.cc:290] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3db910924ef94d0cb36917e6480483e4 (127.15.81.194:39873), 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333)
I20251023 20:15:25.231626 17483 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db910924ef94d0cb36917e6480483e4" is_pre_election: true
I20251023 20:15:25.231628 17407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227661b68e6b438ab50331b3ec09fe54" is_pre_election: true
I20251023 20:15:25.231858 17611 raft_consensus.cc:493] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:15:25.231873 17483 raft_consensus.cc:2468] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 0.
I20251023 20:15:25.231911 17407 raft_consensus.cc:2468] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 0.
I20251023 20:15:25.231989 17611 raft_consensus.cc:515] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.232285 17521 leader_election.cc:304] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3db910924ef94d0cb36917e6480483e4, 6fa99129dc754d238dcd9cb7ac9d2b57; no voters:
I20251023 20:15:25.232517 17611 leader_election.cc:290] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333), 3db910924ef94d0cb36917e6480483e4 (127.15.81.194:39873)
I20251023 20:15:25.232658 17611 raft_consensus.cc:2804] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:25.232704 17611 raft_consensus.cc:493] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:25.232741 17611 raft_consensus.cc:3060] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:25.232949 17407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "819a8d5d8dec402d8d750cc1778f93d3" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227661b68e6b438ab50331b3ec09fe54" is_pre_election: true
I20251023 20:15:25.233096 17407 raft_consensus.cc:2468] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 0.
I20251023 20:15:25.233407 17524 leader_election.cc:304] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227661b68e6b438ab50331b3ec09fe54, 6fa99129dc754d238dcd9cb7ac9d2b57; no voters:
I20251023 20:15:25.233485 17483 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "819a8d5d8dec402d8d750cc1778f93d3" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db910924ef94d0cb36917e6480483e4" is_pre_election: true
I20251023 20:15:25.233626 17617 raft_consensus.cc:2804] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:15:25.233644 17483 raft_consensus.cc:2468] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 0.
I20251023 20:15:25.233717 17617 raft_consensus.cc:493] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:15:25.233779 17617 raft_consensus.cc:3060] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:25.307489 17611 raft_consensus.cc:515] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.307626 17617 raft_consensus.cc:515] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.308269 17617 leader_election.cc:290] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 election: Requested vote from peers 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333), 3db910924ef94d0cb36917e6480483e4 (127.15.81.194:39873)
I20251023 20:15:25.308293 17611 leader_election.cc:290] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 election: Requested vote from peers 3db910924ef94d0cb36917e6480483e4 (127.15.81.194:39873), 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333)
I20251023 20:15:25.308951 17407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "819a8d5d8dec402d8d750cc1778f93d3" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227661b68e6b438ab50331b3ec09fe54"
I20251023 20:15:25.309026 17483 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db910924ef94d0cb36917e6480483e4"
I20251023 20:15:25.309121 17407 raft_consensus.cc:3060] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:25.309180 17483 raft_consensus.cc:3060] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:25.309136 17406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227661b68e6b438ab50331b3ec09fe54"
I20251023 20:15:25.309298 17406 raft_consensus.cc:3060] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:25.309037 17482 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "819a8d5d8dec402d8d750cc1778f93d3" candidate_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db910924ef94d0cb36917e6480483e4"
I20251023 20:15:25.312410 17482 raft_consensus.cc:3060] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:25.406364 17406 raft_consensus.cc:2468] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 1.
I20251023 20:15:25.406569 17407 raft_consensus.cc:2468] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 1.
I20251023 20:15:25.406364 17483 raft_consensus.cc:2468] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 1.
I20251023 20:15:25.407081 17524 leader_election.cc:304] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227661b68e6b438ab50331b3ec09fe54, 6fa99129dc754d238dcd9cb7ac9d2b57; no voters:
I20251023 20:15:25.407349 17521 leader_election.cc:304] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3db910924ef94d0cb36917e6480483e4, 6fa99129dc754d238dcd9cb7ac9d2b57; no voters:
I20251023 20:15:25.407685 17617 raft_consensus.cc:2804] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:25.407794 17611 raft_consensus.cc:2804] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:25.409309 17482 raft_consensus.cc:2468] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fa99129dc754d238dcd9cb7ac9d2b57 in term 1.
I20251023 20:15:25.409778 17611 raft_consensus.cc:697] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Becoming Leader. State: Replica: 6fa99129dc754d238dcd9cb7ac9d2b57, State: Running, Role: LEADER
I20251023 20:15:25.410382 17617 raft_consensus.cc:697] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Becoming Leader. State: Replica: 6fa99129dc754d238dcd9cb7ac9d2b57, State: Running, Role: LEADER
I20251023 20:15:25.410369 17611 consensus_queue.cc:237] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.410617 17617 consensus_queue.cc:237] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } }
I20251023 20:15:25.413537 17302 catalog_manager.cc:5649] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fa99129dc754d238dcd9cb7ac9d2b57 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } health_report { overall_health: HEALTHY } } }
I20251023 20:15:25.413929 17302 catalog_manager.cc:5649] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fa99129dc754d238dcd9cb7ac9d2b57 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:25.769790 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:25.776551 17558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66"
dest_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57"
mode: GRACEFUL
new_leader_uuid: "3db910924ef94d0cb36917e6480483e4"
from {username='slave'} at 127.0.0.1:42482
I20251023 20:15:25.776749 17558 raft_consensus.cc:606] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Received request to transfer leadership to TS 3db910924ef94d0cb36917e6480483e4
I20251023 20:15:25.777269 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:15:25.777376 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:25.836179 17611 consensus_queue.cc:1048] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:25.838811 17617 raft_consensus.cc:993] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: : Instructing follower 3db910924ef94d0cb36917e6480483e4 to start an election
I20251023 20:15:25.839020 17617 raft_consensus.cc:1081] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Signalling peer 3db910924ef94d0cb36917e6480483e4 to start an election
I20251023 20:15:25.839732 17483 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66"
dest_uuid: "3db910924ef94d0cb36917e6480483e4"
from {username='slave'} at 127.0.0.1:43596
I20251023 20:15:25.840708 17483 raft_consensus.cc:493] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:15:25.840811 17483 raft_consensus.cc:3060] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:25.841277 17619 consensus_queue.cc:1048] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:25.865929 17617 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:25.869812 17619 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:15:25.911605 17483 raft_consensus.cc:515] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.912576 17483 leader_election.cc:290] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [CANDIDATE]: Term 2 election: Requested vote from peers 6fa99129dc754d238dcd9cb7ac9d2b57 (127.15.81.195:43645), 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333)
I20251023 20:15:25.919947 17407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66" candidate_uuid: "3db910924ef94d0cb36917e6480483e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "227661b68e6b438ab50331b3ec09fe54"
I20251023 20:15:25.920136 17407 raft_consensus.cc:3060] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:25.920136 17558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f70fefdea4842d0b538316d35440d66" candidate_uuid: "3db910924ef94d0cb36917e6480483e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57"
I20251023 20:15:25.920327 17558 raft_consensus.cc:3055] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:15:25.920396 17558 raft_consensus.cc:740] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6fa99129dc754d238dcd9cb7ac9d2b57, State: Running, Role: LEADER
I20251023 20:15:25.920665 17558 consensus_queue.cc:260] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:25.921006 17558 raft_consensus.cc:3060] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:15:26.016988 17558 raft_consensus.cc:2468] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3db910924ef94d0cb36917e6480483e4 in term 2.
I20251023 20:15:26.017040 17407 raft_consensus.cc:2468] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3db910924ef94d0cb36917e6480483e4 in term 2.
I20251023 20:15:26.017599 17448 leader_election.cc:304] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227661b68e6b438ab50331b3ec09fe54, 3db910924ef94d0cb36917e6480483e4; no voters:
I20251023 20:15:26.017906 17624 raft_consensus.cc:2804] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:15:26.020226 17624 raft_consensus.cc:697] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 2 LEADER]: Becoming Leader. State: Replica: 3db910924ef94d0cb36917e6480483e4, State: Running, Role: LEADER
I20251023 20:15:26.020510 17624 consensus_queue.cc:237] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } }
I20251023 20:15:26.022746 17302 catalog_manager.cc:5649] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 reported cstate change: term changed from 1 to 2, leader changed from 6fa99129dc754d238dcd9cb7ac9d2b57 (127.15.81.195) to 3db910924ef94d0cb36917e6480483e4 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "3db910924ef94d0cb36917e6480483e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20251023 20:15:26.616801 17558 raft_consensus.cc:1275] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 2 FOLLOWER]: Refusing update from remote peer 3db910924ef94d0cb36917e6480483e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:15:26.617404 17643 consensus_queue.cc:1048] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:26.620720 17407 raft_consensus.cc:1275] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 2 FOLLOWER]: Refusing update from remote peer 3db910924ef94d0cb36917e6480483e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:15:26.621227 17644 consensus_queue.cc:1048] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:27.777650 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:27.778167 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:27.778275 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:29.778609 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:29.779110 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:29.779196 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:31.779582 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:31.780014 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:31.780112 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:33.780510 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:33.781311 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:33.781510 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:35.781908 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:35.782470 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:35.782608 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:36.535321 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:36.538441 17653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:36.538731 17654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:36.538861 17656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:36.539868 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:36.540618 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:36.540695 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:36.540738 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250536540724 us; error 0 us; skew 500 ppm
I20251023 20:15:36.540887 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:36.542147 15687 webserver.cc:492] Webserver started at http://127.15.81.196:41683/ using document root <none> and password file <none>
I20251023 20:15:36.542377 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:36.542452 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:36.542603 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:36.543187 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "e601534945fa4563aea1865038132934"
format_stamp: "Formatted at 2025-10-23 20:15:36 on dist-test-slave-jr94"
I20251023 20:15:36.545719 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:15:36.547219 17661 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:36.547606 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:15:36.547710 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "e601534945fa4563aea1865038132934"
format_stamp: "Formatted at 2025-10-23 20:15:36 on dist-test-slave-jr94"
I20251023 20:15:36.547806 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:36.562278 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:36.562815 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:36.563299 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:15:36.564028 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:15:36.564102 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:36.564167 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:15:36.564224 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:36.578825 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:40595
I20251023 20:15:36.578866 17723 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:40595 every 8 connection(s)
I20251023 20:15:36.589380 17724 heartbeater.cc:344] Connected to a master server at 127.15.81.254:40855
I20251023 20:15:36.589519 17724 heartbeater.cc:461] Registering TS with master...
I20251023 20:15:36.589870 17724 heartbeater.cc:507] Master 127.15.81.254:40855 requested a full tablet report, sending...
I20251023 20:15:36.590632 17302 ts_manager.cc:194] Registered new tserver with Master: e601534945fa4563aea1865038132934 (127.15.81.196:40595)
I20251023 20:15:36.591373 17302 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55806
I20251023 20:15:36.790642 17558 consensus_queue.cc:237] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } }
I20251023 20:15:36.792912 17407 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:36.793783 17652 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:36.799988 17483 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:15:36.800791 17731 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e601534945fa4563aea1865038132934 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } } }
I20251023 20:15:36.805255 17735 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:15:36.806881 17407 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e601534945fa4563aea1865038132934 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } } }
I20251023 20:15:36.807943 17483 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e601534945fa4563aea1865038132934 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } } }
I20251023 20:15:36.813831 17304 catalog_manager.cc:5649] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e601534945fa4563aea1865038132934 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } } }
W20251023 20:15:36.815641 17523 consensus_peers.cc:597] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 -> Peer e601534945fa4563aea1865038132934 (127.15.81.196:40595): Couldn't send request to peer e601534945fa4563aea1865038132934. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 819a8d5d8dec402d8d750cc1778f93d3. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:15:37.407205 17746 ts_tablet_manager.cc:933] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: Initiating tablet copy from peer 6fa99129dc754d238dcd9cb7ac9d2b57 (127.15.81.195:43645)
I20251023 20:15:37.408183 17746 tablet_copy_client.cc:323] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:43645
I20251023 20:15:37.414965 17568 tablet_copy_service.cc:140] P 6fa99129dc754d238dcd9cb7ac9d2b57: Received BeginTabletCopySession request for tablet 819a8d5d8dec402d8d750cc1778f93d3 from peer e601534945fa4563aea1865038132934 ({username='slave'} at 127.0.0.1:34608)
I20251023 20:15:37.415122 17568 tablet_copy_service.cc:161] P 6fa99129dc754d238dcd9cb7ac9d2b57: Beginning new tablet copy session on tablet 819a8d5d8dec402d8d750cc1778f93d3 from peer e601534945fa4563aea1865038132934 at {username='slave'} at 127.0.0.1:34608: session id = e601534945fa4563aea1865038132934-819a8d5d8dec402d8d750cc1778f93d3
I20251023 20:15:37.418033 17568 tablet_copy_source_session.cc:215] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:15:37.419881 17746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819a8d5d8dec402d8d750cc1778f93d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:37.425506 17746 tablet_copy_client.cc:806] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: tablet copy: Starting download of 0 data blocks...
I20251023 20:15:37.425776 17746 tablet_copy_client.cc:670] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: tablet copy: Starting download of 1 WAL segments...
I20251023 20:15:37.427690 17746 tablet_copy_client.cc:538] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:15:37.431182 17746 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: Bootstrap starting.
I20251023 20:15:37.438458 17746 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:15:37.438843 17746 tablet_bootstrap.cc:492] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: Bootstrap complete.
I20251023 20:15:37.439092 17746 ts_tablet_manager.cc:1403] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20251023 20:15:37.439739 17746 raft_consensus.cc:359] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } }
I20251023 20:15:37.439949 17746 raft_consensus.cc:740] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e601534945fa4563aea1865038132934, State: Initialized, Role: LEARNER
I20251023 20:15:37.440147 17746 consensus_queue.cc:260] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: true } }
I20251023 20:15:37.440804 17724 heartbeater.cc:499] Master 127.15.81.254:40855 was elected leader, sending a full tablet report...
I20251023 20:15:37.440888 17746 ts_tablet_manager.cc:1434] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251023 20:15:37.441603 17568 tablet_copy_service.cc:342] P 6fa99129dc754d238dcd9cb7ac9d2b57: Request end of tablet copy session e601534945fa4563aea1865038132934-819a8d5d8dec402d8d750cc1778f93d3 received from {username='slave'} at 127.0.0.1:34608
I20251023 20:15:37.441772 17568 tablet_copy_service.cc:434] P 6fa99129dc754d238dcd9cb7ac9d2b57: ending tablet copy session e601534945fa4563aea1865038132934-819a8d5d8dec402d8d750cc1778f93d3 on tablet 819a8d5d8dec402d8d750cc1778f93d3 with peer e601534945fa4563aea1865038132934
I20251023 20:15:37.782933 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:37.783479 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:37.783610 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:37.854101 17699 raft_consensus.cc:1217] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 20:15:38.469453 17745 raft_consensus.cc:1064] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: attempting to promote NON_VOTER e601534945fa4563aea1865038132934 to VOTER
I20251023 20:15:38.470342 17745 consensus_queue.cc:237] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } }
I20251023 20:15:38.474256 17699 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 LEARNER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:15:38.474278 17483 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:15:38.474876 17407 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:15:38.474948 17754 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:15:38.475373 17752 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:15:38.475996 17745 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:15:38.481227 17752 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e601534945fa4563aea1865038132934 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.484247 17483 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e601534945fa4563aea1865038132934 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.486678 17302 catalog_manager.cc:5649] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 reported cstate change: config changed from index 2 to 3, e601534945fa4563aea1865038132934 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:15:38.487782 17699 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e601534945fa4563aea1865038132934 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.496611 17406 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e601534945fa4563aea1865038132934 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227661b68e6b438ab50331b3ec09fe54" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 42333 } attrs { replace: true } } peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.500504 17557 consensus_queue.cc:237] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } }
I20251023 20:15:38.502548 17699 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 FOLLOWER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:15:38.502604 17483 raft_consensus.cc:1275] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Refusing update from remote peer 6fa99129dc754d238dcd9cb7ac9d2b57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:15:38.503180 17752 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:15:38.505429 17752 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.506105 17699 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.509223 17289 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 819a8d5d8dec402d8d750cc1778f93d3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:15:38.510306 17754 consensus_queue.cc:1048] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:15:38.510250 17304 catalog_manager.cc:5649] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 reported cstate change: config changed from index 3 to 4, VOTER 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:15:38.511766 17483 raft_consensus.cc:2955] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3db910924ef94d0cb36917e6480483e4" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 39873 } } peers { permanent_uuid: "6fa99129dc754d238dcd9cb7ac9d2b57" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 43645 } } peers { permanent_uuid: "e601534945fa4563aea1865038132934" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 40595 } attrs { promote: false } } }
I20251023 20:15:38.514384 17397 tablet_service.cc:1552] Processing DeleteTablet for tablet 819a8d5d8dec402d8d750cc1778f93d3 with delete_type TABLET_DATA_TOMBSTONED (TS 227661b68e6b438ab50331b3ec09fe54 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33176
I20251023 20:15:38.518851 17764 tablet_replica.cc:333] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: stopping tablet replica
I20251023 20:15:38.519026 17764 raft_consensus.cc:2243] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:38.519275 17764 raft_consensus.cc:2272] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:38.520165 17764 ts_tablet_manager.cc:1916] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:15:38.524088 17764 ts_tablet_manager.cc:1929] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251023 20:15:38.524192 17764 log.cc:1199] T 819a8d5d8dec402d8d750cc1778f93d3 P 227661b68e6b438ab50331b3ec09fe54: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/819a8d5d8dec402d8d750cc1778f93d3
I20251023 20:15:38.524770 17291 catalog_manager.cc:4985] TS 227661b68e6b438ab50331b3ec09fe54 (127.15.81.193:42333): tablet 819a8d5d8dec402d8d750cc1778f93d3 (table test-workload [id=26cb338297e54356aa0eb4105730b808]) successfully deleted
I20251023 20:15:39.784019 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:39.784556 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:39.784646 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:41.785056 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:41.785610 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:41.785704 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:43.786168 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:43.786804 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:43.786895 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:45.787177 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:45.787683 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:45.787770 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:47.788126 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:47.788699 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:47.788791 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:49.789175 17357 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:15:49.789779 17357 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:15:49.789872 17357 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:15:49.937726 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:15:49.948812 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:49.949149 15687 tablet_replica.cc:333] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54: stopping tablet replica
I20251023 20:15:49.949360 15687 raft_consensus.cc:2243] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:49.949538 15687 raft_consensus.cc:2272] T 9f70fefdea4842d0b538316d35440d66 P 227661b68e6b438ab50331b3ec09fe54 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:49.965997 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:15:49.969120 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:15:49.978314 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:49.978560 15687 tablet_replica.cc:333] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4: stopping tablet replica
I20251023 20:15:49.978734 15687 raft_consensus.cc:2243] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:49.978946 15687 raft_consensus.cc:2272] T 819a8d5d8dec402d8d750cc1778f93d3 P 3db910924ef94d0cb36917e6480483e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:49.979493 15687 tablet_replica.cc:333] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4: stopping tablet replica
I20251023 20:15:49.979602 15687 raft_consensus.cc:2243] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:15:49.979849 15687 raft_consensus.cc:2272] T 9f70fefdea4842d0b538316d35440d66 P 3db910924ef94d0cb36917e6480483e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:49.993448 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:15:49.996327 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:15:50.004455 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:50.004673 15687 tablet_replica.cc:333] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57: stopping tablet replica
I20251023 20:15:50.004829 15687 raft_consensus.cc:2243] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:50.005029 15687 raft_consensus.cc:2272] T 9f70fefdea4842d0b538316d35440d66 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:50.005548 15687 tablet_replica.cc:333] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57: stopping tablet replica
I20251023 20:15:50.005700 15687 raft_consensus.cc:2243] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:50.005967 15687 raft_consensus.cc:2272] T 819a8d5d8dec402d8d750cc1778f93d3 P 6fa99129dc754d238dcd9cb7ac9d2b57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:50.020938 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:15:50.023823 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:15:50.032025 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:15:50.032248 15687 tablet_replica.cc:333] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934: stopping tablet replica
I20251023 20:15:50.032399 15687 raft_consensus.cc:2243] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:15:50.032656 15687 raft_consensus.cc:2272] T 819a8d5d8dec402d8d750cc1778f93d3 P e601534945fa4563aea1865038132934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:50.045591 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:15:50.048224 15687 master.cc:561] Master@127.15.81.254:40855 shutting down...
I20251023 20:15:50.059880 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:15:50.060086 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:15:50.060176 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0a426e1d837b4699b4ca4841d58cbd4b: stopping tablet replica
I20251023 20:15:50.073117 15687 master.cc:583] Master@127.15.81.254:40855 shutdown complete.
[ OK ] AutoRebalancerTest.MovesScheduledIfAddTserver (26384 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTservers
I20251023 20:15:50.083895 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:34859
I20251023 20:15:50.084345 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:15:50.086203 17770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:50.086385 17771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:15:50.086833 17773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:15:50.087857 15687 server_base.cc:1047] running on GCE node
I20251023 20:15:50.088074 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:15:50.088124 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:15:50.088164 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250550088160 us; error 0 us; skew 500 ppm
I20251023 20:15:50.088300 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:15:50.089466 15687 webserver.cc:492] Webserver started at http://127.15.81.254:43721/ using document root <none> and password file <none>
I20251023 20:15:50.089671 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:15:50.089727 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:15:50.089859 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:15:50.090472 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "8ed07aa67cc0434fa1ac1c85848ea531"
format_stamp: "Formatted at 2025-10-23 20:15:50 on dist-test-slave-jr94"
I20251023 20:15:50.092928 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:15:50.094342 17778 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:15:50.094739 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:15:50.094820 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "8ed07aa67cc0434fa1ac1c85848ea531"
format_stamp: "Formatted at 2025-10-23 20:15:50 on dist-test-slave-jr94"
I20251023 20:15:50.094900 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:15:50.114857 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:15:50.115309 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:15:50.126300 17829 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:34859 every 8 connection(s)
I20251023 20:15:50.126281 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:34859
I20251023 20:15:50.127543 17830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:15:50.131285 17830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531: Bootstrap starting.
I20251023 20:15:50.132355 17830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531: Neither blocks nor log segments found. Creating new log.
I20251023 20:15:50.133733 17830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531: No bootstrap required, opened a new log
I20251023 20:15:50.134315 17830 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" member_type: VOTER }
I20251023 20:15:50.134440 17830 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:15:50.134493 17830 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed07aa67cc0434fa1ac1c85848ea531, State: Initialized, Role: FOLLOWER
I20251023 20:15:50.134678 17830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" member_type: VOTER }
I20251023 20:15:50.134778 17830 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:15:50.134835 17830 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:15:50.134910 17830 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:15:50.136452 17830 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" member_type: VOTER }
I20251023 20:15:50.136616 17830 leader_election.cc:304] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ed07aa67cc0434fa1ac1c85848ea531; no voters:
I20251023 20:15:50.136991 17830 leader_election.cc:290] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:15:50.137075 17833 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:15:50.137562 17833 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 1 LEADER]: Becoming Leader. State: Replica: 8ed07aa67cc0434fa1ac1c85848ea531, State: Running, Role: LEADER
I20251023 20:15:50.137756 17833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" member_type: VOTER }
I20251023 20:15:50.137820 17830 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:15:50.138449 17834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" member_type: VOTER } }
I20251023 20:15:50.138509 17835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ed07aa67cc0434fa1ac1c85848ea531. Latest consensus state: current_term: 1 leader_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed07aa67cc0434fa1ac1c85848ea531" member_type: VOTER } }
I20251023 20:15:50.138736 17835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:50.139217 17834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [sys.catalog]: This master's current role is: LEADER
I20251023 20:15:50.139779 17839 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:15:50.141019 17839 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:15:50.142653 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:15:50.143679 17839 catalog_manager.cc:1357] Generated new cluster ID: 83c6c0be00ab4020bb7b27cd91935430
I20251023 20:15:50.143755 17839 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:15:50.160188 17839 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:15:50.160956 17839 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:15:50.166294 17839 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531: Generated new TSK 0
I20251023 20:15:50.166503 17839 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:15:50.174619 15687 internal_mini_cluster.cc:371] 0 TS(s) registered with all masters after 5.1075e-05s
I20251023 20:16:02.257014 15687 master.cc:561] Master@127.15.81.254:34859 shutting down...
I20251023 20:16:02.268463 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:16:02.268697 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:02.268795 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8ed07aa67cc0434fa1ac1c85848ea531: stopping tablet replica
I20251023 20:16:02.283497 15687 master.cc:583] Master@127.15.81.254:34859 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTservers (12207 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTablets
I20251023 20:16:02.291188 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:34541
I20251023 20:16:02.291672 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:02.293697 17852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.293728 17853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.294315 17855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:02.297498 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:02.298518 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:02.298590 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:02.298631 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250562298618 us; error 0 us; skew 500 ppm
I20251023 20:16:02.298753 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:02.299789 15687 webserver.cc:492] Webserver started at http://127.15.81.254:37457/ using document root <none> and password file <none>
I20251023 20:16:02.299976 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:02.300041 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:02.300144 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:02.300745 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "4378b08e421f4b9494db3ad93cfcc430"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.303146 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20251023 20:16:02.304582 17860 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.304947 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:02.305045 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "4378b08e421f4b9494db3ad93cfcc430"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.305166 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:02.313472 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:02.313894 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:02.323598 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:34541
I20251023 20:16:02.323628 17911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:34541 every 8 connection(s)
I20251023 20:16:02.324739 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:02.328482 17912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430: Bootstrap starting.
I20251023 20:16:02.329586 17912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:02.330879 17912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430: No bootstrap required, opened a new log
I20251023 20:16:02.331425 17912 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4378b08e421f4b9494db3ad93cfcc430" member_type: VOTER }
I20251023 20:16:02.331552 17912 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:02.331627 17912 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4378b08e421f4b9494db3ad93cfcc430, State: Initialized, Role: FOLLOWER
I20251023 20:16:02.331826 17912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4378b08e421f4b9494db3ad93cfcc430" member_type: VOTER }
I20251023 20:16:02.331944 17912 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:16:02.332010 17912 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:16:02.332089 17912 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:02.333757 17912 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4378b08e421f4b9494db3ad93cfcc430" member_type: VOTER }
I20251023 20:16:02.333942 17912 leader_election.cc:304] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4378b08e421f4b9494db3ad93cfcc430; no voters:
I20251023 20:16:02.334280 17912 leader_election.cc:290] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:16:02.334372 17915 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:02.334858 17915 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 1 LEADER]: Becoming Leader. State: Replica: 4378b08e421f4b9494db3ad93cfcc430, State: Running, Role: LEADER
I20251023 20:16:02.335083 17912 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:16:02.335065 17915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4378b08e421f4b9494db3ad93cfcc430" member_type: VOTER }
I20251023 20:16:02.336259 17916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4378b08e421f4b9494db3ad93cfcc430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4378b08e421f4b9494db3ad93cfcc430" member_type: VOTER } }
I20251023 20:16:02.336287 17917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4378b08e421f4b9494db3ad93cfcc430. Latest consensus state: current_term: 1 leader_uuid: "4378b08e421f4b9494db3ad93cfcc430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4378b08e421f4b9494db3ad93cfcc430" member_type: VOTER } }
I20251023 20:16:02.336587 17916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:02.336617 17917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:02.337610 17921 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:16:02.338737 17921 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:16:02.341468 17921 catalog_manager.cc:1357] Generated new cluster ID: 9be18f5c1a6d4f448ae7d81192b61a40
I20251023 20:16:02.341562 17921 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:16:02.343861 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:16:02.355772 17921 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:16:02.356489 17921 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:16:02.367409 17921 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430: Generated new TSK 0
I20251023 20:16:02.367607 17921 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:16:02.376346 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:02.379014 17933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.379328 17934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:02.379875 15687 server_base.cc:1047] running on GCE node
W20251023 20:16:02.379884 17936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:02.380193 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:02.380256 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:02.380296 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250562380283 us; error 0 us; skew 500 ppm
I20251023 20:16:02.380422 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:02.381562 15687 webserver.cc:492] Webserver started at http://127.15.81.193:32847/ using document root <none> and password file <none>
I20251023 20:16:02.381764 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:02.381827 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:02.381932 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:02.382508 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "f3af30dbbb794e0997cce2485d683673"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.384876 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:02.386240 17941 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.386544 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:02.386636 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "f3af30dbbb794e0997cce2485d683673"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.386731 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:02.398633 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:02.399073 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:02.399552 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:02.400285 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:02.400354 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.400419 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:02.400465 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.412096 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:38755
I20251023 20:16:02.412123 18003 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:38755 every 8 connection(s)
I20251023 20:16:02.413861 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:02.416466 18008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.416977 18009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.417503 18011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:02.418231 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:02.418530 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:02.418627 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:02.418694 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250562418669 us; error 0 us; skew 500 ppm
I20251023 20:16:02.418908 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:02.420399 15687 webserver.cc:492] Webserver started at http://127.15.81.194:35525/ using document root <none> and password file <none>
I20251023 20:16:02.420624 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:02.420711 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:02.420778 18004 heartbeater.cc:344] Connected to a master server at 127.15.81.254:34541
I20251023 20:16:02.420886 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:02.420948 18004 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:02.421325 18004 heartbeater.cc:507] Master 127.15.81.254:34541 requested a full tablet report, sending...
I20251023 20:16:02.421615 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "457ee9ea211c4034a93fa9ab215e9530"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.422159 17877 ts_manager.cc:194] Registered new tserver with Master: f3af30dbbb794e0997cce2485d683673 (127.15.81.193:38755)
I20251023 20:16:02.423135 17877 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45912
I20251023 20:16:02.424160 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:02.425634 18016 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.425925 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:02.426030 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "457ee9ea211c4034a93fa9ab215e9530"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.426131 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:02.432345 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:02.432758 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:02.433245 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:02.433981 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:02.434052 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.434147 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:02.434203 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.446236 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:42757
I20251023 20:16:02.446264 18079 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:42757 every 8 connection(s)
I20251023 20:16:02.447664 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:02.450307 18083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.450459 18084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:02.454418 18086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:02.454921 18080 heartbeater.cc:344] Connected to a master server at 127.15.81.254:34541
I20251023 20:16:02.455051 18080 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:02.455227 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:02.455440 18080 heartbeater.cc:507] Master 127.15.81.254:34541 requested a full tablet report, sending...
I20251023 20:16:02.455513 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:02.455600 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:02.455665 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250562455642 us; error 0 us; skew 500 ppm
I20251023 20:16:02.455809 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:02.456385 17877 ts_manager.cc:194] Registered new tserver with Master: 457ee9ea211c4034a93fa9ab215e9530 (127.15.81.194:42757)
I20251023 20:16:02.457084 15687 webserver.cc:492] Webserver started at http://127.15.81.195:36309/ using document root <none> and password file <none>
I20251023 20:16:02.457329 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:02.457432 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:02.457477 17877 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45922
I20251023 20:16:02.457576 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:02.458230 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "6abbbe4469c44435bafdfdd5bc7ea17e"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.460434 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:02.461865 18091 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.462188 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:02.462291 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "6abbbe4469c44435bafdfdd5bc7ea17e"
format_stamp: "Formatted at 2025-10-23 20:16:02 on dist-test-slave-jr94"
I20251023 20:16:02.462400 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:02.484251 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:02.484781 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:02.485339 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:02.486080 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:02.486147 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.486251 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:02.486311 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:02.498286 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:34007
I20251023 20:16:02.498327 18154 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:34007 every 8 connection(s)
I20251023 20:16:02.504382 18155 heartbeater.cc:344] Connected to a master server at 127.15.81.254:34541
I20251023 20:16:02.504513 18155 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:02.504801 18155 heartbeater.cc:507] Master 127.15.81.254:34541 requested a full tablet report, sending...
I20251023 20:16:02.505513 17877 ts_manager.cc:194] Registered new tserver with Master: 6abbbe4469c44435bafdfdd5bc7ea17e (127.15.81.195:34007)
I20251023 20:16:02.505825 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006716467s
I20251023 20:16:02.506170 17877 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45934
I20251023 20:16:03.424438 18004 heartbeater.cc:499] Master 127.15.81.254:34541 was elected leader, sending a full tablet report...
I20251023 20:16:03.458623 18080 heartbeater.cc:499] Master 127.15.81.254:34541 was elected leader, sending a full tablet report...
I20251023 20:16:03.507289 18155 heartbeater.cc:499] Master 127.15.81.254:34541 was elected leader, sending a full tablet report...
I20251023 20:16:04.343719 17931 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:06.344043 17931 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:08.344587 17931 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:10.345017 17931 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:12.345528 17931 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:14.345999 17931 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:14.595651 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:16:14.610581 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:14.624858 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:16:14.627349 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:16:14.635478 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:14.650480 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:16:14.652863 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:16:14.661370 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:14.673776 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:16:14.676046 15687 master.cc:561] Master@127.15.81.254:34541 shutting down...
I20251023 20:16:14.683560 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:16:14.683769 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:14.683890 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4378b08e421f4b9494db3ad93cfcc430: stopping tablet replica
I20251023 20:16:14.696732 15687 master.cc:583] Master@127.15.81.254:34541 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTablets (12414 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne
I20251023 20:16:14.705768 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:35667
I20251023 20:16:14.706245 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:14.708398 18159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.708613 18160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.709233 15687 server_base.cc:1047] running on GCE node
W20251023 20:16:14.709326 18162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.709658 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:14.709730 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:14.709800 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250574709775 us; error 0 us; skew 500 ppm
I20251023 20:16:14.709957 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:14.710985 15687 webserver.cc:492] Webserver started at http://127.15.81.254:33847/ using document root <none> and password file <none>
I20251023 20:16:14.711184 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:14.711261 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:14.711400 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:14.712024 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "0ffd10aab3064ee48f1e01a5b57cedcb"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.714421 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:14.715754 18167 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.716050 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:14.716167 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "0ffd10aab3064ee48f1e01a5b57cedcb"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.716277 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:14.724676 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:14.725230 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:14.736068 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:35667
I20251023 20:16:14.736090 18219 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:35667 every 8 connection(s)
I20251023 20:16:14.737277 18220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:14.740891 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb: Bootstrap starting.
I20251023 20:16:14.742077 18220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:14.743520 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb: No bootstrap required, opened a new log
I20251023 20:16:14.744020 18220 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" member_type: VOTER }
I20251023 20:16:14.744140 18220 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:14.744191 18220 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd10aab3064ee48f1e01a5b57cedcb, State: Initialized, Role: FOLLOWER
I20251023 20:16:14.744402 18220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" member_type: VOTER }
I20251023 20:16:14.744522 18220 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:16:14.744577 18220 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:16:14.744638 18220 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:14.746147 18220 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" member_type: VOTER }
I20251023 20:16:14.746309 18220 leader_election.cc:304] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ffd10aab3064ee48f1e01a5b57cedcb; no voters:
I20251023 20:16:14.746641 18220 leader_election.cc:290] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:16:14.746767 18223 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:14.747215 18223 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 1 LEADER]: Becoming Leader. State: Replica: 0ffd10aab3064ee48f1e01a5b57cedcb, State: Running, Role: LEADER
I20251023 20:16:14.747439 18220 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:16:14.747432 18223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" member_type: VOTER }
I20251023 20:16:14.748561 18224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" member_type: VOTER } }
I20251023 20:16:14.748787 18224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:14.748607 18225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ffd10aab3064ee48f1e01a5b57cedcb. Latest consensus state: current_term: 1 leader_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffd10aab3064ee48f1e01a5b57cedcb" member_type: VOTER } }
I20251023 20:16:14.749145 18225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:14.749516 18229 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:16:14.750986 18229 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:16:14.752497 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:16:14.753978 18229 catalog_manager.cc:1357] Generated new cluster ID: a8c7442816974fe79611a56f2f6e4111
I20251023 20:16:14.754065 18229 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:16:14.766170 18229 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:16:14.766844 18229 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:16:14.786576 18229 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb: Generated new TSK 0
I20251023 20:16:14.786821 18229 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:16:14.817803 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:14.820290 18241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.820996 18242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.821125 18244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.821406 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:14.821647 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:14.821707 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:14.821748 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250574821734 us; error 0 us; skew 500 ppm
I20251023 20:16:14.821887 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:14.822930 15687 webserver.cc:492] Webserver started at http://127.15.81.193:33401/ using document root <none> and password file <none>
I20251023 20:16:14.823144 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:14.823213 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:14.823320 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:14.823880 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "7f9af724e47247d18057edf087058e01"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.826236 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:16:14.827664 18249 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.827951 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:14.828052 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "7f9af724e47247d18057edf087058e01"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.828153 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:14.835669 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:14.836158 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:14.836669 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:14.837503 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:14.837574 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.837683 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:14.837743 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.850332 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:35951
I20251023 20:16:14.850355 18312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:35951 every 8 connection(s)
I20251023 20:16:14.852015 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:14.854763 18317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.858430 18318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.859016 18320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.859243 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:14.859462 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:14.859513 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:14.859553 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250574859538 us; error 0 us; skew 500 ppm
I20251023 20:16:14.859651 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:14.860641 15687 webserver.cc:492] Webserver started at http://127.15.81.194:44295/ using document root <none> and password file <none>
I20251023 20:16:14.860821 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:14.860915 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:14.861023 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:14.861577 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "01ee81273d2d4f8db347b1fbe5ae1762"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.863310 18313 heartbeater.cc:344] Connected to a master server at 127.15.81.254:35667
I20251023 20:16:14.863442 18313 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:14.863768 18313 heartbeater.cc:507] Master 127.15.81.254:35667 requested a full tablet report, sending...
I20251023 20:16:14.864369 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:16:14.864638 18185 ts_manager.cc:194] Registered new tserver with Master: 7f9af724e47247d18057edf087058e01 (127.15.81.193:35951)
I20251023 20:16:14.865662 18185 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52504
I20251023 20:16:14.865990 18325 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.866395 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:14.866503 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "01ee81273d2d4f8db347b1fbe5ae1762"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.866612 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:14.879983 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:14.880475 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:14.880985 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:14.881778 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:14.881850 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.881953 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:14.882010 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.893599 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:42751
I20251023 20:16:14.893625 18388 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:42751 every 8 connection(s)
I20251023 20:16:14.895124 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:14.897812 18392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.898104 18393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.899516 15687 server_base.cc:1047] running on GCE node
W20251023 20:16:14.899955 18395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.900373 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:14.900451 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:14.900503 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250574900483 us; error 0 us; skew 500 ppm
I20251023 20:16:14.900691 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:14.901969 15687 webserver.cc:492] Webserver started at http://127.15.81.195:45509/ using document root <none> and password file <none>
I20251023 20:16:14.902166 18389 heartbeater.cc:344] Connected to a master server at 127.15.81.254:35667
I20251023 20:16:14.902202 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:14.902294 18389 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:14.902608 18389 heartbeater.cc:507] Master 127.15.81.254:35667 requested a full tablet report, sending...
I20251023 20:16:14.903460 18185 ts_manager.cc:194] Registered new tserver with Master: 01ee81273d2d4f8db347b1fbe5ae1762 (127.15.81.194:42751)
I20251023 20:16:14.904367 18185 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52510
I20251023 20:16:14.910923 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:14.911039 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:14.911520 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "426c177d0ab6400787d5d8bb15bcd994"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.913713 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20251023 20:16:14.914948 18400 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.915230 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:14.915309 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "426c177d0ab6400787d5d8bb15bcd994"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.915403 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:14.921162 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:14.921566 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:14.921981 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:14.922660 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:14.922714 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.922777 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:14.922814 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.932337 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:44351
I20251023 20:16:14.932354 18463 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:44351 every 8 connection(s)
I20251023 20:16:14.933717 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:14.935784 18467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:14.935994 18468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.937253 15687 server_base.cc:1047] running on GCE node
W20251023 20:16:14.937305 18470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:14.937645 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:14.937700 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:14.937721 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250574937719 us; error 0 us; skew 500 ppm
I20251023 20:16:14.937829 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:14.939342 15687 webserver.cc:492] Webserver started at http://127.15.81.196:36319/ using document root <none> and password file <none>
I20251023 20:16:14.939563 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:14.939637 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:14.939662 18464 heartbeater.cc:344] Connected to a master server at 127.15.81.254:35667
I20251023 20:16:14.939744 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:14.939782 18464 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:14.940052 18464 heartbeater.cc:507] Master 127.15.81.254:35667 requested a full tablet report, sending...
I20251023 20:16:14.940285 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "e7efc28d03874e5f9876c3fe566de5e1"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.940749 18185 ts_manager.cc:194] Registered new tserver with Master: 426c177d0ab6400787d5d8bb15bcd994 (127.15.81.195:44351)
I20251023 20:16:14.941452 18185 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52520
I20251023 20:16:14.942636 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:14.943960 18475 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.944286 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:14.944361 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "e7efc28d03874e5f9876c3fe566de5e1"
format_stamp: "Formatted at 2025-10-23 20:16:14 on dist-test-slave-jr94"
I20251023 20:16:14.944446 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:14.953227 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:14.953554 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:14.953933 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:14.954561 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:14.954607 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.954667 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:14.954694 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:14.965004 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:36413
I20251023 20:16:14.965029 18537 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:36413 every 8 connection(s)
I20251023 20:16:14.970798 18538 heartbeater.cc:344] Connected to a master server at 127.15.81.254:35667
I20251023 20:16:14.970894 18538 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:14.971163 18538 heartbeater.cc:507] Master 127.15.81.254:35667 requested a full tablet report, sending...
I20251023 20:16:14.971885 18185 ts_manager.cc:194] Registered new tserver with Master: e7efc28d03874e5f9876c3fe566de5e1 (127.15.81.196:36413)
I20251023 20:16:14.972399 15687 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.006644975s
I20251023 20:16:14.972582 18185 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52536
I20251023 20:16:15.867182 18313 heartbeater.cc:499] Master 127.15.81.254:35667 was elected leader, sending a full tablet report...
I20251023 20:16:15.905429 18389 heartbeater.cc:499] Master 127.15.81.254:35667 was elected leader, sending a full tablet report...
I20251023 20:16:15.942615 18464 heartbeater.cc:499] Master 127.15.81.254:35667 was elected leader, sending a full tablet report...
I20251023 20:16:15.973809 18538 heartbeater.cc:499] Master 127.15.81.254:35667 was elected leader, sending a full tablet report...
I20251023 20:16:16.006564 15687 test_util.cc:276] Using random seed: 452033940
I20251023 20:16:16.016873 18185 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52560:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251023 20:16:16.039542 18354 tablet_service.cc:1505] Processing CreateTablet for tablet aebb887be8ae4889847c7815ad1af99d (DEFAULT_TABLE table=test-workload [id=7635b888f0554f17aba125ab4ae98e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:16:16.039947 18354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aebb887be8ae4889847c7815ad1af99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:16.039995 18278 tablet_service.cc:1505] Processing CreateTablet for tablet aebb887be8ae4889847c7815ad1af99d (DEFAULT_TABLE table=test-workload [id=7635b888f0554f17aba125ab4ae98e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:16:16.040334 18278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aebb887be8ae4889847c7815ad1af99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:16.044862 18503 tablet_service.cc:1505] Processing CreateTablet for tablet aebb887be8ae4889847c7815ad1af99d (DEFAULT_TABLE table=test-workload [id=7635b888f0554f17aba125ab4ae98e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:16:16.045181 18503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aebb887be8ae4889847c7815ad1af99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:16.050025 18561 tablet_bootstrap.cc:492] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: Bootstrap starting.
I20251023 20:16:16.050025 18562 tablet_bootstrap.cc:492] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1: Bootstrap starting.
I20251023 20:16:16.050428 18560 tablet_bootstrap.cc:492] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762: Bootstrap starting.
I20251023 20:16:16.051762 18562 tablet_bootstrap.cc:654] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:16.051755 18561 tablet_bootstrap.cc:654] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:16.053532 18560 tablet_bootstrap.cc:654] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:16.054399 18562 tablet_bootstrap.cc:492] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1: No bootstrap required, opened a new log
I20251023 20:16:16.054584 18562 ts_tablet_manager.cc:1403] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251023 20:16:16.054678 18561 tablet_bootstrap.cc:492] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: No bootstrap required, opened a new log
I20251023 20:16:16.054811 18561 ts_tablet_manager.cc:1403] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251023 20:16:16.055179 18560 tablet_bootstrap.cc:492] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762: No bootstrap required, opened a new log
I20251023 20:16:16.055291 18560 ts_tablet_manager.cc:1403] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:16:16.055245 18562 raft_consensus.cc:359] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.055433 18562 raft_consensus.cc:385] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:16.055524 18562 raft_consensus.cc:740] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7efc28d03874e5f9876c3fe566de5e1, State: Initialized, Role: FOLLOWER
I20251023 20:16:16.055544 18561 raft_consensus.cc:359] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.055706 18561 raft_consensus.cc:385] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:16.055806 18561 raft_consensus.cc:740] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f9af724e47247d18057edf087058e01, State: Initialized, Role: FOLLOWER
I20251023 20:16:16.055787 18562 consensus_queue.cc:260] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.055994 18560 raft_consensus.cc:359] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.056010 18561 consensus_queue.cc:260] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.056159 18560 raft_consensus.cc:385] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:16.056212 18560 raft_consensus.cc:740] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ee81273d2d4f8db347b1fbe5ae1762, State: Initialized, Role: FOLLOWER
I20251023 20:16:16.056428 18560 consensus_queue.cc:260] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.057340 18560 ts_tablet_manager.cc:1434] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:16.062821 18562 ts_tablet_manager.cc:1434] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20251023 20:16:16.063373 18561 ts_tablet_manager.cc:1434] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: Time spent starting tablet: real 0.008s user 0.002s sys 0.006s
I20251023 20:16:16.072644 18568 raft_consensus.cc:493] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:16.072755 18568 raft_consensus.cc:515] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.073532 18568 leader_election.cc:290] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f9af724e47247d18057edf087058e01 (127.15.81.193:35951), e7efc28d03874e5f9876c3fe566de5e1 (127.15.81.196:36413)
I20251023 20:16:16.079578 18288 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aebb887be8ae4889847c7815ad1af99d" candidate_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f9af724e47247d18057edf087058e01" is_pre_election: true
I20251023 20:16:16.079667 18513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aebb887be8ae4889847c7815ad1af99d" candidate_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7efc28d03874e5f9876c3fe566de5e1" is_pre_election: true
I20251023 20:16:16.079788 18288 raft_consensus.cc:2468] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ee81273d2d4f8db347b1fbe5ae1762 in term 0.
I20251023 20:16:16.079824 18513 raft_consensus.cc:2468] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ee81273d2d4f8db347b1fbe5ae1762 in term 0.
I20251023 20:16:16.080140 18327 leader_election.cc:304] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ee81273d2d4f8db347b1fbe5ae1762, e7efc28d03874e5f9876c3fe566de5e1; no voters:
I20251023 20:16:16.080359 18568 raft_consensus.cc:2804] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:16.080428 18568 raft_consensus.cc:493] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:16.080469 18568 raft_consensus.cc:3060] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:16.082275 18568 raft_consensus.cc:515] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.082716 18568 leader_election.cc:290] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [CANDIDATE]: Term 1 election: Requested vote from peers 7f9af724e47247d18057edf087058e01 (127.15.81.193:35951), e7efc28d03874e5f9876c3fe566de5e1 (127.15.81.196:36413)
I20251023 20:16:16.083151 18513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aebb887be8ae4889847c7815ad1af99d" candidate_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7efc28d03874e5f9876c3fe566de5e1"
I20251023 20:16:16.083115 18288 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aebb887be8ae4889847c7815ad1af99d" candidate_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f9af724e47247d18057edf087058e01"
I20251023 20:16:16.083302 18513 raft_consensus.cc:3060] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:16.083336 18288 raft_consensus.cc:3060] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:16.085808 18288 raft_consensus.cc:2468] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ee81273d2d4f8db347b1fbe5ae1762 in term 1.
I20251023 20:16:16.085808 18513 raft_consensus.cc:2468] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ee81273d2d4f8db347b1fbe5ae1762 in term 1.
I20251023 20:16:16.086190 18327 leader_election.cc:304] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ee81273d2d4f8db347b1fbe5ae1762, 7f9af724e47247d18057edf087058e01; no voters:
I20251023 20:16:16.086396 18568 raft_consensus.cc:2804] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:16.086743 18568 raft_consensus.cc:697] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 1 LEADER]: Becoming Leader. State: Replica: 01ee81273d2d4f8db347b1fbe5ae1762, State: Running, Role: LEADER
I20251023 20:16:16.086947 18568 consensus_queue.cc:237] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } }
I20251023 20:16:16.089109 18185 catalog_manager.cc:5649] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ee81273d2d4f8db347b1fbe5ae1762 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ee81273d2d4f8db347b1fbe5ae1762" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 42751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:16.102208 18314 tablet.cc:2378] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 20:16:16.507138 18568 consensus_queue.cc:1048] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7efc28d03874e5f9876c3fe566de5e1" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:16.512092 18568 consensus_queue.cc:1048] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f9af724e47247d18057edf087058e01" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 35951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:16.752501 18238 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:16.752944 18238 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:16.753057 18238 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:18.753405 18238 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:18.753865 18238 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:18.753955 18238 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:20.754454 18238 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:20.754884 18238 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:20.754976 18238 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:22.755396 18238 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:22.755851 18238 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:22.755959 18238 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:24.756366 18238 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:24.756806 18238 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:24.756928 18238 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:26.757294 18238 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:26.757706 18238 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:26.757783 18238 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:27.145985 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:16:27.157866 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:27.158126 15687 tablet_replica.cc:333] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01: stopping tablet replica
I20251023 20:16:27.158313 15687 raft_consensus.cc:2243] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:27.158516 15687 raft_consensus.cc:2272] T aebb887be8ae4889847c7815ad1af99d P 7f9af724e47247d18057edf087058e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:27.172415 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:16:27.175254 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:16:27.183745 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:27.183991 15687 tablet_replica.cc:333] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762: stopping tablet replica
I20251023 20:16:27.184158 15687 raft_consensus.cc:2243] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:16:27.184468 15687 raft_consensus.cc:2272] T aebb887be8ae4889847c7815ad1af99d P 01ee81273d2d4f8db347b1fbe5ae1762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:27.197911 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:16:27.200416 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:16:27.207755 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:27.220170 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:16:27.222586 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:16:27.231142 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:27.231395 15687 tablet_replica.cc:333] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1: stopping tablet replica
I20251023 20:16:27.231555 15687 raft_consensus.cc:2243] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:27.231719 15687 raft_consensus.cc:2272] T aebb887be8ae4889847c7815ad1af99d P e7efc28d03874e5f9876c3fe566de5e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:27.244875 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:16:27.256817 15687 master.cc:561] Master@127.15.81.254:35667 shutting down...
I20251023 20:16:27.263684 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:16:27.263898 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:27.263998 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0ffd10aab3064ee48f1e01a5b57cedcb: stopping tablet replica
I20251023 20:16:27.277174 15687 master.cc:583] Master@127.15.81.254:35667 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne (12581 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy
I20251023 20:16:27.287181 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:46493
I20251023 20:16:27.287645 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:27.289867 18582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.289862 18583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.290167 18585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:27.290297 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:27.290637 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:27.290688 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:27.290709 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250587290705 us; error 0 us; skew 500 ppm
I20251023 20:16:27.290851 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:27.291900 15687 webserver.cc:492] Webserver started at http://127.15.81.254:37547/ using document root <none> and password file <none>
I20251023 20:16:27.292094 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:27.292152 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:27.292291 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:27.292920 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "be22c287999f42e2b5f32643eb004636"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.295265 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 20:16:27.296581 18590 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.296890 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:27.296985 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "be22c287999f42e2b5f32643eb004636"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.297082 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:27.315837 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:27.316282 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:27.326714 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:46493
I20251023 20:16:27.326737 18641 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:46493 every 8 connection(s)
I20251023 20:16:27.327841 18642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:27.331604 18642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636: Bootstrap starting.
I20251023 20:16:27.332702 18642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:27.334034 18642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636: No bootstrap required, opened a new log
I20251023 20:16:27.334589 18642 raft_consensus.cc:359] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be22c287999f42e2b5f32643eb004636" member_type: VOTER }
I20251023 20:16:27.334704 18642 raft_consensus.cc:385] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:27.334756 18642 raft_consensus.cc:740] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be22c287999f42e2b5f32643eb004636, State: Initialized, Role: FOLLOWER
I20251023 20:16:27.334944 18642 consensus_queue.cc:260] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be22c287999f42e2b5f32643eb004636" member_type: VOTER }
I20251023 20:16:27.335041 18642 raft_consensus.cc:399] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:16:27.335100 18642 raft_consensus.cc:493] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:16:27.335176 18642 raft_consensus.cc:3060] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:27.336778 18642 raft_consensus.cc:515] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be22c287999f42e2b5f32643eb004636" member_type: VOTER }
I20251023 20:16:27.336982 18642 leader_election.cc:304] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be22c287999f42e2b5f32643eb004636; no voters:
I20251023 20:16:27.337334 18642 leader_election.cc:290] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:16:27.337456 18645 raft_consensus.cc:2804] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:27.337917 18645 raft_consensus.cc:697] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 1 LEADER]: Becoming Leader. State: Replica: be22c287999f42e2b5f32643eb004636, State: Running, Role: LEADER
I20251023 20:16:27.338114 18645 consensus_queue.cc:237] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be22c287999f42e2b5f32643eb004636" member_type: VOTER }
I20251023 20:16:27.338160 18642 sys_catalog.cc:565] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:16:27.338785 18647 sys_catalog.cc:455] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be22c287999f42e2b5f32643eb004636. Latest consensus state: current_term: 1 leader_uuid: "be22c287999f42e2b5f32643eb004636" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be22c287999f42e2b5f32643eb004636" member_type: VOTER } }
I20251023 20:16:27.338887 18646 sys_catalog.cc:455] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be22c287999f42e2b5f32643eb004636" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be22c287999f42e2b5f32643eb004636" member_type: VOTER } }
I20251023 20:16:27.339068 18647 sys_catalog.cc:458] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:27.339146 18646 sys_catalog.cc:458] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:27.340138 18650 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:16:27.341629 18650 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:16:27.343122 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:16:27.344291 18650 catalog_manager.cc:1357] Generated new cluster ID: 918e9de7d34e4073a7adbff739812fc9
I20251023 20:16:27.344363 18650 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:16:27.357676 18650 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:16:27.358309 18650 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:16:27.366847 18650 catalog_manager.cc:6022] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636: Generated new TSK 0
I20251023 20:16:27.367009 18650 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:16:27.375663 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:27.377843 18663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.378114 18664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:27.378638 15687 server_base.cc:1047] running on GCE node
W20251023 20:16:27.378683 18666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:27.378976 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:27.379030 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:27.379051 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250587379049 us; error 0 us; skew 500 ppm
I20251023 20:16:27.379141 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:27.380076 15687 webserver.cc:492] Webserver started at http://127.15.81.193:36783/ using document root <none> and password file <none>
I20251023 20:16:27.380297 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:27.380367 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:27.380453 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:27.381057 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "991dffabc680411cac42f5d694980898"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.383363 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20251023 20:16:27.384886 18671 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.385250 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:27.385340 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "991dffabc680411cac42f5d694980898"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.385432 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:27.410168 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:27.410606 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:27.411068 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:27.411835 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:27.411885 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.411939 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:27.411967 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.423168 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:37467
I20251023 20:16:27.423189 18734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:37467 every 8 connection(s)
I20251023 20:16:27.424638 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:27.427080 18740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.427381 18741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.428402 18743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:27.429082 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:27.429294 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:27.429348 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:27.429422 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250587429417 us; error 0 us; skew 500 ppm
I20251023 20:16:27.429565 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:27.430728 15687 webserver.cc:492] Webserver started at http://127.15.81.194:41311/ using document root <none> and password file <none>
I20251023 20:16:27.430927 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:27.430994 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:27.431102 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:27.431650 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "a8114ffcd70b4385993b657889f68c41"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.434074 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:27.434196 18736 heartbeater.cc:344] Connected to a master server at 127.15.81.254:46493
I20251023 20:16:27.434304 18736 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:27.434665 18736 heartbeater.cc:507] Master 127.15.81.254:46493 requested a full tablet report, sending...
I20251023 20:16:27.435452 18607 ts_manager.cc:194] Registered new tserver with Master: 991dffabc680411cac42f5d694980898 (127.15.81.193:37467)
I20251023 20:16:27.435766 18748 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.436075 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:27.436167 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "a8114ffcd70b4385993b657889f68c41"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.436280 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:27.436808 18607 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40496
I20251023 20:16:27.446384 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:27.446736 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:27.447157 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:27.447857 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:27.447911 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.447970 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:27.448005 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.457968 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:43593
I20251023 20:16:27.457993 18810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:43593 every 8 connection(s)
I20251023 20:16:27.459268 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:27.461436 18814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.462450 18815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:27.462917 18818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:27.463022 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:27.463421 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:27.463492 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:27.463553 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250587463549 us; error 0 us; skew 500 ppm
I20251023 20:16:27.463764 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:27.465196 15687 webserver.cc:492] Webserver started at http://127.15.81.195:46441/ using document root <none> and password file <none>
I20251023 20:16:27.465441 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:27.465521 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:27.465628 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:27.465728 18811 heartbeater.cc:344] Connected to a master server at 127.15.81.254:46493
I20251023 20:16:27.465831 18811 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:27.466091 18811 heartbeater.cc:507] Master 127.15.81.254:46493 requested a full tablet report, sending...
I20251023 20:16:27.466261 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "65c4d5879f6941a0bcc2d136c5fbf549"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.466864 18607 ts_manager.cc:194] Registered new tserver with Master: a8114ffcd70b4385993b657889f68c41 (127.15.81.194:43593)
I20251023 20:16:27.467795 18607 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40500
I20251023 20:16:27.469033 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:16:27.470355 18823 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.470636 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:27.470748 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "65c4d5879f6941a0bcc2d136c5fbf549"
format_stamp: "Formatted at 2025-10-23 20:16:27 on dist-test-slave-jr94"
I20251023 20:16:27.470853 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:27.477244 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:27.477644 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:27.478093 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:27.478816 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:27.478886 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.478983 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:27.479055 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:27.489463 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:38381
I20251023 20:16:27.489490 18888 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:38381 every 8 connection(s)
I20251023 20:16:27.495393 18889 heartbeater.cc:344] Connected to a master server at 127.15.81.254:46493
I20251023 20:16:27.495527 18889 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:27.495847 18889 heartbeater.cc:507] Master 127.15.81.254:46493 requested a full tablet report, sending...
I20251023 20:16:27.496542 18607 ts_manager.cc:194] Registered new tserver with Master: 65c4d5879f6941a0bcc2d136c5fbf549 (127.15.81.195:38381)
I20251023 20:16:27.496953 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006718212s
I20251023 20:16:27.497470 18607 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40516
I20251023 20:16:28.437999 18736 heartbeater.cc:499] Master 127.15.81.254:46493 was elected leader, sending a full tablet report...
I20251023 20:16:28.468807 18811 heartbeater.cc:499] Master 127.15.81.254:46493 was elected leader, sending a full tablet report...
I20251023 20:16:28.498755 18889 heartbeater.cc:499] Master 127.15.81.254:46493 was elected leader, sending a full tablet report...
I20251023 20:16:28.530344 15687 test_util.cc:276] Using random seed: 464557729
I20251023 20:16:28.546020 18607 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40540:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:16:28.546548 18607 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:16:28.561721 18776 tablet_service.cc:1505] Processing CreateTablet for tablet f81bbfee93ca4889ab768130afed2afc (DEFAULT_TABLE table=test-workload [id=c8e799fbb2184d2e82a4b010f95971ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:16:28.561939 18853 tablet_service.cc:1505] Processing CreateTablet for tablet f81bbfee93ca4889ab768130afed2afc (DEFAULT_TABLE table=test-workload [id=c8e799fbb2184d2e82a4b010f95971ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:16:28.562209 18776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81bbfee93ca4889ab768130afed2afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:28.562299 18853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81bbfee93ca4889ab768130afed2afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:28.565481 18699 tablet_service.cc:1505] Processing CreateTablet for tablet f81bbfee93ca4889ab768130afed2afc (DEFAULT_TABLE table=test-workload [id=c8e799fbb2184d2e82a4b010f95971ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:16:28.565852 18699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81bbfee93ca4889ab768130afed2afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:28.568596 18910 tablet_bootstrap.cc:492] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549: Bootstrap starting.
I20251023 20:16:28.570397 18910 tablet_bootstrap.cc:654] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:28.572187 18913 tablet_bootstrap.cc:492] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898: Bootstrap starting.
I20251023 20:16:28.572281 18910 tablet_bootstrap.cc:492] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549: No bootstrap required, opened a new log
I20251023 20:16:28.572444 18910 ts_tablet_manager.cc:1403] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:16:28.572942 18911 tablet_bootstrap.cc:492] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41: Bootstrap starting.
I20251023 20:16:28.573180 18910 raft_consensus.cc:359] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.573407 18910 raft_consensus.cc:385] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:28.573518 18910 raft_consensus.cc:740] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c4d5879f6941a0bcc2d136c5fbf549, State: Initialized, Role: FOLLOWER
I20251023 20:16:28.573689 18913 tablet_bootstrap.cc:654] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:28.573776 18910 consensus_queue.cc:260] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.574586 18910 ts_tablet_manager.cc:1434] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:28.575331 18911 tablet_bootstrap.cc:654] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:28.576614 18913 tablet_bootstrap.cc:492] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898: No bootstrap required, opened a new log
I20251023 20:16:28.576749 18913 ts_tablet_manager.cc:1403] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:16:28.576912 18911 tablet_bootstrap.cc:492] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41: No bootstrap required, opened a new log
I20251023 20:16:28.577093 18911 ts_tablet_manager.cc:1403] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:16:28.577520 18913 raft_consensus.cc:359] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.577719 18913 raft_consensus.cc:385] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:28.577661 18911 raft_consensus.cc:359] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.577811 18913 raft_consensus.cc:740] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991dffabc680411cac42f5d694980898, State: Initialized, Role: FOLLOWER
I20251023 20:16:28.577844 18911 raft_consensus.cc:385] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:28.577931 18911 raft_consensus.cc:740] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8114ffcd70b4385993b657889f68c41, State: Initialized, Role: FOLLOWER
I20251023 20:16:28.578056 18913 consensus_queue.cc:260] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.578190 18911 consensus_queue.cc:260] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.578886 18913 ts_tablet_manager.cc:1434] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:28.578989 18911 ts_tablet_manager.cc:1434] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:16:28.591099 18914 raft_consensus.cc:493] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:28.591229 18914 raft_consensus.cc:515] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.591964 18914 leader_election.cc:290] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8114ffcd70b4385993b657889f68c41 (127.15.81.194:43593), 991dffabc680411cac42f5d694980898 (127.15.81.193:37467)
I20251023 20:16:28.598834 18786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f81bbfee93ca4889ab768130afed2afc" candidate_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8114ffcd70b4385993b657889f68c41" is_pre_election: true
I20251023 20:16:28.598834 18710 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f81bbfee93ca4889ab768130afed2afc" candidate_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991dffabc680411cac42f5d694980898" is_pre_election: true
I20251023 20:16:28.599123 18786 raft_consensus.cc:2468] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65c4d5879f6941a0bcc2d136c5fbf549 in term 0.
I20251023 20:16:28.599123 18710 raft_consensus.cc:2468] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65c4d5879f6941a0bcc2d136c5fbf549 in term 0.
I20251023 20:16:28.599586 18827 leader_election.cc:304] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 65c4d5879f6941a0bcc2d136c5fbf549, 991dffabc680411cac42f5d694980898, a8114ffcd70b4385993b657889f68c41; no voters:
I20251023 20:16:28.599835 18914 raft_consensus.cc:2804] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:28.599918 18914 raft_consensus.cc:493] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:28.600003 18914 raft_consensus.cc:3060] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:28.602114 18914 raft_consensus.cc:515] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.602608 18914 leader_election.cc:290] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [CANDIDATE]: Term 1 election: Requested vote from peers a8114ffcd70b4385993b657889f68c41 (127.15.81.194:43593), 991dffabc680411cac42f5d694980898 (127.15.81.193:37467)
I20251023 20:16:28.602948 18786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f81bbfee93ca4889ab768130afed2afc" candidate_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8114ffcd70b4385993b657889f68c41"
I20251023 20:16:28.603003 18710 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f81bbfee93ca4889ab768130afed2afc" candidate_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991dffabc680411cac42f5d694980898"
I20251023 20:16:28.603093 18786 raft_consensus.cc:3060] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:28.603178 18710 raft_consensus.cc:3060] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:28.604883 18786 raft_consensus.cc:2468] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65c4d5879f6941a0bcc2d136c5fbf549 in term 1.
I20251023 20:16:28.604969 18710 raft_consensus.cc:2468] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65c4d5879f6941a0bcc2d136c5fbf549 in term 1.
I20251023 20:16:28.605227 18826 leader_election.cc:304] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65c4d5879f6941a0bcc2d136c5fbf549, a8114ffcd70b4385993b657889f68c41; no voters:
I20251023 20:16:28.605490 18914 raft_consensus.cc:2804] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:28.605916 18914 raft_consensus.cc:697] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 1 LEADER]: Becoming Leader. State: Replica: 65c4d5879f6941a0bcc2d136c5fbf549, State: Running, Role: LEADER
I20251023 20:16:28.606154 18914 consensus_queue.cc:237] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } }
I20251023 20:16:28.608040 18605 catalog_manager.cc:5649] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65c4d5879f6941a0bcc2d136c5fbf549 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c4d5879f6941a0bcc2d136c5fbf549" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 38381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:29.034339 18914 consensus_queue.cc:1048] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8114ffcd70b4385993b657889f68c41" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:29.038730 18914 consensus_queue.cc:1048] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dffabc680411cac42f5d694980898" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:29.342782 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:29.343143 18660 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:29.343585 18660 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:29.343703 18660 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:30.345459 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:31.344110 18660 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:31.344592 18660 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:31.344717 18660 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:31.347648 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:32.350057 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:33.345175 18660 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:33.345636 18660 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:33.345755 18660 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:33.352371 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:34.354658 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:35.346163 18660 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:35.346655 18660 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:35.346786 18660 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:35.357012 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:36.359280 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:37.347257 18660 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:37.347766 18660 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:37.347884 18660 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:37.361853 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:38.364382 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:39.348393 18660 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:39.348820 18660 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:39.348953 18660 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:39.366715 18655 placement_policy_util.cc:407] tablet f81bbfee93ca4889ab768130afed2afc: detected majority of replicas (2 of 3) at location L0
I20251023 20:16:39.673293 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:16:39.692106 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:39.692381 15687 tablet_replica.cc:333] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898: stopping tablet replica
I20251023 20:16:39.692564 15687 raft_consensus.cc:2243] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:39.692754 15687 raft_consensus.cc:2272] T f81bbfee93ca4889ab768130afed2afc P 991dffabc680411cac42f5d694980898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:39.705945 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:16:39.708420 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:16:39.715456 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:39.715734 15687 tablet_replica.cc:333] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41: stopping tablet replica
I20251023 20:16:39.715899 15687 raft_consensus.cc:2243] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:39.716048 15687 raft_consensus.cc:2272] T f81bbfee93ca4889ab768130afed2afc P a8114ffcd70b4385993b657889f68c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:39.729301 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:16:39.731907 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:16:39.739128 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:16:39.739352 15687 tablet_replica.cc:333] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549: stopping tablet replica
I20251023 20:16:39.739490 15687 raft_consensus.cc:2243] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:16:39.739750 15687 raft_consensus.cc:2272] T f81bbfee93ca4889ab768130afed2afc P 65c4d5879f6941a0bcc2d136c5fbf549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:39.753069 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:16:39.755571 15687 master.cc:561] Master@127.15.81.254:46493 shutting down...
I20251023 20:16:39.762094 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:16:39.762284 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:39.762373 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P be22c287999f42e2b5f32643eb004636: stopping tablet replica
I20251023 20:16:39.775204 15687 master.cc:583] Master@127.15.81.254:46493 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy (12497 ms)
[ RUN ] AutoRebalancerTest.TestMaxMovesPerServer
I20251023 20:16:39.784943 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:45023
I20251023 20:16:39.785392 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:39.787683 18929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:39.787706 18930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:39.787706 18932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:39.787951 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:39.788354 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:39.788414 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:39.788444 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250599788442 us; error 0 us; skew 500 ppm
I20251023 20:16:39.788590 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:39.789739 15687 webserver.cc:492] Webserver started at http://127.15.81.254:41439/ using document root <none> and password file <none>
I20251023 20:16:39.789944 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:39.790005 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:39.790124 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:39.790732 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "e3bc3766d3674c358cb246e89a6996df"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.793053 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 20:16:39.794381 18937 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.794710 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:39.794806 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "e3bc3766d3674c358cb246e89a6996df"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.794900 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:39.808697 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:39.809299 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:39.819254 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:45023
I20251023 20:16:39.819283 18988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:45023 every 8 connection(s)
I20251023 20:16:39.820398 18989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:39.824081 18989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df: Bootstrap starting.
I20251023 20:16:39.825393 18989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:39.826781 18989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df: No bootstrap required, opened a new log
I20251023 20:16:39.827283 18989 raft_consensus.cc:359] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3bc3766d3674c358cb246e89a6996df" member_type: VOTER }
I20251023 20:16:39.827397 18989 raft_consensus.cc:385] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:39.827433 18989 raft_consensus.cc:740] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3bc3766d3674c358cb246e89a6996df, State: Initialized, Role: FOLLOWER
I20251023 20:16:39.827579 18989 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3bc3766d3674c358cb246e89a6996df" member_type: VOTER }
I20251023 20:16:39.827684 18989 raft_consensus.cc:399] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:16:39.827723 18989 raft_consensus.cc:493] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:16:39.827771 18989 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:39.829322 18989 raft_consensus.cc:515] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3bc3766d3674c358cb246e89a6996df" member_type: VOTER }
I20251023 20:16:39.829471 18989 leader_election.cc:304] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3bc3766d3674c358cb246e89a6996df; no voters:
I20251023 20:16:39.829767 18989 leader_election.cc:290] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:16:39.829877 18992 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:39.830297 18992 raft_consensus.cc:697] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 1 LEADER]: Becoming Leader. State: Replica: e3bc3766d3674c358cb246e89a6996df, State: Running, Role: LEADER
I20251023 20:16:39.830498 18992 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3bc3766d3674c358cb246e89a6996df" member_type: VOTER }
I20251023 20:16:39.830540 18989 sys_catalog.cc:565] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:16:39.831254 18994 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3bc3766d3674c358cb246e89a6996df. Latest consensus state: current_term: 1 leader_uuid: "e3bc3766d3674c358cb246e89a6996df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3bc3766d3674c358cb246e89a6996df" member_type: VOTER } }
I20251023 20:16:39.831301 18993 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3bc3766d3674c358cb246e89a6996df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3bc3766d3674c358cb246e89a6996df" member_type: VOTER } }
I20251023 20:16:39.831384 18994 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:39.831415 18993 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [sys.catalog]: This master's current role is: LEADER
I20251023 20:16:39.833679 18997 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:16:39.835892 18997 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:16:39.839002 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:16:39.839159 18997 catalog_manager.cc:1357] Generated new cluster ID: fabc956544fb42f29f6a9ce3db3d02dd
I20251023 20:16:39.839226 18997 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:16:39.860399 18997 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:16:39.861141 18997 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:16:39.871470 18997 catalog_manager.cc:6022] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df: Generated new TSK 0
I20251023 20:16:39.871662 18997 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:16:39.903635 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:39.906165 19010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:39.906371 19011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:39.907563 15687 server_base.cc:1047] running on GCE node
W20251023 20:16:39.907598 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:39.907948 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:39.908003 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:39.908041 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250599908038 us; error 0 us; skew 500 ppm
I20251023 20:16:39.908166 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:39.909304 15687 webserver.cc:492] Webserver started at http://127.15.81.193:34341/ using document root <none> and password file <none>
I20251023 20:16:39.909541 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:39.909619 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:39.909746 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:39.910365 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "116b80452edd43bc9d822d31be0c5c38"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.912587 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:39.913962 19018 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.914258 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:39.914342 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "116b80452edd43bc9d822d31be0c5c38"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.914443 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:39.922129 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:39.922623 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:39.923101 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:39.923863 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:39.923916 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.923996 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:39.924036 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.935676 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:36987
I20251023 20:16:39.935712 19081 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:36987 every 8 connection(s)
I20251023 20:16:39.937247 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:39.939847 19086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:39.940167 19089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:39.940728 19087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:39.941099 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:39.941372 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:39.941430 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:39.941465 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250599941461 us; error 0 us; skew 500 ppm
I20251023 20:16:39.941653 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:39.943076 15687 webserver.cc:492] Webserver started at http://127.15.81.194:33223/ using document root <none> and password file <none>
I20251023 20:16:39.943320 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:39.943400 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:39.943516 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:39.944003 19082 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45023
I20251023 20:16:39.944093 19082 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:39.944171 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "f909677d13e94318ad9a3b1b2db05723"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.944378 19082 heartbeater.cc:507] Master 127.15.81.254:45023 requested a full tablet report, sending...
I20251023 20:16:39.945250 18954 ts_manager.cc:194] Registered new tserver with Master: 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:39.946230 18954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35012
I20251023 20:16:39.946614 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:16:39.947924 19094 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.948220 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:39.948297 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "f909677d13e94318ad9a3b1b2db05723"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.948391 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:39.963946 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:39.964447 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:39.964984 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:39.965741 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:39.965792 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.965876 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:39.965924 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.977421 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:33259
I20251023 20:16:39.977438 19156 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:33259 every 8 connection(s)
I20251023 20:16:39.978852 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:39.981199 19161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:39.981158 19160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:39.986550 19157 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45023
I20251023 20:16:39.986708 19157 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:39.987129 19157 heartbeater.cc:507] Master 127.15.81.254:45023 requested a full tablet report, sending...
W20251023 20:16:39.987828 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:39.988114 18954 ts_manager.cc:194] Registered new tserver with Master: f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:39.989647 18954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35016
I20251023 20:16:39.993753 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:39.993976 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:39.994030 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:39.994072 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250599994067 us; error 0 us; skew 500 ppm
I20251023 20:16:39.994202 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:39.995304 15687 webserver.cc:492] Webserver started at http://127.15.81.195:41363/ using document root <none> and password file <none>
I20251023 20:16:39.995500 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:39.995563 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:39.995688 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:39.996245 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "cea0984de40845da94e6f23513009f02"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:39.998314 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:16:39.999550 19168 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:39.999831 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:16:39.999915 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "cea0984de40845da94e6f23513009f02"
format_stamp: "Formatted at 2025-10-23 20:16:39 on dist-test-slave-jr94"
I20251023 20:16:40.000000 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:40.015659 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:40.016165 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:40.016625 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:40.017390 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:40.017442 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:40.017518 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:40.017553 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:40.027132 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:37799
I20251023 20:16:40.027156 19230 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:37799 every 8 connection(s)
I20251023 20:16:40.032691 19231 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45023
I20251023 20:16:40.032799 19231 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:40.033066 19231 heartbeater.cc:507] Master 127.15.81.254:45023 requested a full tablet report, sending...
I20251023 20:16:40.033794 18954 ts_manager.cc:194] Registered new tserver with Master: cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:40.034452 18954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35020
I20251023 20:16:40.034790 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006845269s
I20251023 20:16:40.947430 19082 heartbeater.cc:499] Master 127.15.81.254:45023 was elected leader, sending a full tablet report...
I20251023 20:16:40.991400 19157 heartbeater.cc:499] Master 127.15.81.254:45023 was elected leader, sending a full tablet report...
I20251023 20:16:41.035542 19231 heartbeater.cc:499] Master 127.15.81.254:45023 was elected leader, sending a full tablet report...
I20251023 20:16:41.062288 15687 test_util.cc:276] Using random seed: 477089654
I20251023 20:16:41.074083 18954 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35028:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu""\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:16:41.074827 18954 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:16:41.096781 19122 tablet_service.cc:1505] Processing CreateTablet for tablet 2a9aada4eefa459ea9801d5506a2b82e (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251023 20:16:41.097376 19122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9aada4eefa459ea9801d5506a2b82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.101308 19121 tablet_service.cc:1505] Processing CreateTablet for tablet 37a00c71a3934405917c05725c94dde5 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251023 20:16:41.101807 19121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a00c71a3934405917c05725c94dde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.104136 19196 tablet_service.cc:1505] Processing CreateTablet for tablet 2a9aada4eefa459ea9801d5506a2b82e (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251023 20:16:41.104508 19196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9aada4eefa459ea9801d5506a2b82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.105247 19120 tablet_service.cc:1505] Processing CreateTablet for tablet 887ca8402e81417c94da83fdc1b75862 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251023 20:16:41.105738 19120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887ca8402e81417c94da83fdc1b75862. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.109257 19119 tablet_service.cc:1505] Processing CreateTablet for tablet 37f44fa5d63049edbd7ad28d358075fb (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251023 20:16:41.109551 19195 tablet_service.cc:1505] Processing CreateTablet for tablet 37a00c71a3934405917c05725c94dde5 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251023 20:16:41.109691 19119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44fa5d63049edbd7ad28d358075fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.109901 19195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a00c71a3934405917c05725c94dde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.109902 19191 tablet_service.cc:1505] Processing CreateTablet for tablet 38153cd2d04e4c56b22f87e07a3c2d11 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251023 20:16:41.110430 19191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38153cd2d04e4c56b22f87e07a3c2d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.113247 19118 tablet_service.cc:1505] Processing CreateTablet for tablet 461a7ab7c21c40fbb4ef24809dad5831 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251023 20:16:41.113680 19118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461a7ab7c21c40fbb4ef24809dad5831. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.116113 19193 tablet_service.cc:1505] Processing CreateTablet for tablet 37f44fa5d63049edbd7ad28d358075fb (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251023 20:16:41.116576 19193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44fa5d63049edbd7ad28d358075fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.117233 19117 tablet_service.cc:1505] Processing CreateTablet for tablet 38153cd2d04e4c56b22f87e07a3c2d11 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251023 20:16:41.117668 19117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38153cd2d04e4c56b22f87e07a3c2d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.121134 19194 tablet_service.cc:1505] Processing CreateTablet for tablet 887ca8402e81417c94da83fdc1b75862 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251023 20:16:41.121412 19192 tablet_service.cc:1505] Processing CreateTablet for tablet 461a7ab7c21c40fbb4ef24809dad5831 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251023 20:16:41.121526 19194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887ca8402e81417c94da83fdc1b75862. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.121534 19251 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.121817 19192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461a7ab7c21c40fbb4ef24809dad5831. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.123096 19251 tablet_bootstrap.cc:654] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.151315 19120 tablet_service.cc:1505] Processing CreateTablet for tablet 7c0d82f5ba4a476583099f65588d84f0 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251023 20:16:41.151880 19120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0d82f5ba4a476583099f65588d84f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.152249 19194 tablet_service.cc:1505] Processing CreateTablet for tablet fc60b796f0d946fbb49d15812e88ac60 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251023 20:16:41.152583 19194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc60b796f0d946fbb49d15812e88ac60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.153146 19196 tablet_service.cc:1505] Processing CreateTablet for tablet 7184738166284bf3a71cb7c0d38cd863 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251023 20:16:41.154070 19195 tablet_service.cc:1505] Processing CreateTablet for tablet 508756851fc14f93a7d310b11beb15cb (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251023 20:16:41.154382 19195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 508756851fc14f93a7d310b11beb15cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.155122 19253 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.155967 19047 tablet_service.cc:1505] Processing CreateTablet for tablet 2a9aada4eefa459ea9801d5506a2b82e (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251023 20:16:41.156294 19047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9aada4eefa459ea9801d5506a2b82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.156608 19253 tablet_bootstrap.cc:654] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.157544 19196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7184738166284bf3a71cb7c0d38cd863. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.157718 19191 tablet_service.cc:1505] Processing CreateTablet for tablet b4c390009f384f9f9a3f424a69b2e5ce (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251023 20:16:41.151737 19192 tablet_service.cc:1505] Processing CreateTablet for tablet f4ff9ca441b14a468709c0efd0ae984e (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251023 20:16:41.157938 19117 tablet_service.cc:1505] Processing CreateTablet for tablet f4ff9ca441b14a468709c0efd0ae984e (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251023 20:16:41.158085 19191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c390009f384f9f9a3f424a69b2e5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.158275 19117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ff9ca441b14a468709c0efd0ae984e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.158890 19116 tablet_service.cc:1505] Processing CreateTablet for tablet fc60b796f0d946fbb49d15812e88ac60 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251023 20:16:41.159235 19116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc60b796f0d946fbb49d15812e88ac60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.151407 19193 tablet_service.cc:1505] Processing CreateTablet for tablet 7c0d82f5ba4a476583099f65588d84f0 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251023 20:16:41.163475 19115 tablet_service.cc:1505] Processing CreateTablet for tablet 7184738166284bf3a71cb7c0d38cd863 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251023 20:16:41.163854 19115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7184738166284bf3a71cb7c0d38cd863. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.166872 19113 tablet_service.cc:1505] Processing CreateTablet for tablet b4c390009f384f9f9a3f424a69b2e5ce (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251023 20:16:41.167228 19113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c390009f384f9f9a3f424a69b2e5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.167483 19193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0d82f5ba4a476583099f65588d84f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.169173 19114 tablet_service.cc:1505] Processing CreateTablet for tablet 508756851fc14f93a7d310b11beb15cb (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251023 20:16:41.169602 19114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 508756851fc14f93a7d310b11beb15cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.171700 19192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ff9ca441b14a468709c0efd0ae984e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.180878 19043 tablet_service.cc:1505] Processing CreateTablet for tablet 461a7ab7c21c40fbb4ef24809dad5831 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251023 20:16:41.183058 19045 tablet_service.cc:1505] Processing CreateTablet for tablet 887ca8402e81417c94da83fdc1b75862 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251023 20:16:41.185379 19043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461a7ab7c21c40fbb4ef24809dad5831. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.186643 19044 tablet_service.cc:1505] Processing CreateTablet for tablet 37f44fa5d63049edbd7ad28d358075fb (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251023 20:16:41.187192 19039 tablet_service.cc:1505] Processing CreateTablet for tablet fc60b796f0d946fbb49d15812e88ac60 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251023 20:16:41.187822 19046 tablet_service.cc:1505] Processing CreateTablet for tablet 37a00c71a3934405917c05725c94dde5 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251023 20:16:41.188316 19038 tablet_service.cc:1505] Processing CreateTablet for tablet 7184738166284bf3a71cb7c0d38cd863 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251023 20:16:41.188901 19041 tablet_service.cc:1505] Processing CreateTablet for tablet f4ff9ca441b14a468709c0efd0ae984e (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251023 20:16:41.189396 19040 tablet_service.cc:1505] Processing CreateTablet for tablet 7c0d82f5ba4a476583099f65588d84f0 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251023 20:16:41.191879 19045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887ca8402e81417c94da83fdc1b75862. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.191954 19042 tablet_service.cc:1505] Processing CreateTablet for tablet 38153cd2d04e4c56b22f87e07a3c2d11 (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251023 20:16:41.192615 19040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0d82f5ba4a476583099f65588d84f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.195601 19046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a00c71a3934405917c05725c94dde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.196789 19041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ff9ca441b14a468709c0efd0ae984e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.197978 19251 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.198190 19251 ts_tablet_manager.cc:1403] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.077s user 0.032s sys 0.028s
I20251023 20:16:41.197988 19044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44fa5d63049edbd7ad28d358075fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.198895 19251 raft_consensus.cc:359] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.199111 19251 raft_consensus.cc:385] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.199194 19251 raft_consensus.cc:740] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.199291 19042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38153cd2d04e4c56b22f87e07a3c2d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.199450 19251 consensus_queue.cc:260] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.199635 19039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc60b796f0d946fbb49d15812e88ac60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.200999 19038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7184738166284bf3a71cb7c0d38cd863. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.201613 19047 tablet_service.cc:1505] Processing CreateTablet for tablet 508756851fc14f93a7d310b11beb15cb (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251023 20:16:41.201920 19047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 508756851fc14f93a7d310b11beb15cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.203780 19253 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.203936 19253 ts_tablet_manager.cc:1403] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.007s
I20251023 20:16:41.210664 19047 tablet_service.cc:1505] Processing CreateTablet for tablet b4c390009f384f9f9a3f424a69b2e5ce (DEFAULT_TABLE table=test-workload [id=f7183b919b694c279d001a5e3265efa4]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251023 20:16:41.211014 19047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c390009f384f9f9a3f424a69b2e5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:41.224020 19253 raft_consensus.cc:359] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.224172 19253 raft_consensus.cc:385] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.224238 19253 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.224442 19253 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.226629 19255 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.249979 19253 ts_tablet_manager.cc:1434] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.046s user 0.013s sys 0.006s
I20251023 20:16:41.250555 19255 tablet_bootstrap.cc:654] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.252151 19251 ts_tablet_manager.cc:1434] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.054s user 0.013s sys 0.029s
I20251023 20:16:41.252415 19251 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.254269 19251 tablet_bootstrap.cc:654] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.255282 19253 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.256750 19253 tablet_bootstrap.cc:654] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.257089 19255 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.257241 19255 ts_tablet_manager.cc:1403] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.007s
I20251023 20:16:41.258015 19255 raft_consensus.cc:359] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.258183 19255 raft_consensus.cc:385] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.258247 19255 raft_consensus.cc:740] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.258268 19251 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.258387 19253 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.258387 19251 ts_tablet_manager.cc:1403] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20251023 20:16:41.258503 19253 ts_tablet_manager.cc:1403] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251023 20:16:41.258486 19255 consensus_queue.cc:260] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.259084 19251 raft_consensus.cc:359] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.259162 19255 ts_tablet_manager.cc:1434] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:41.259235 19251 raft_consensus.cc:385] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.259153 19253 raft_consensus.cc:359] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.259289 19251 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.259321 19253 raft_consensus.cc:385] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.259383 19253 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.259488 19251 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.259625 19253 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.259917 19251 ts_tablet_manager.cc:1434] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:41.260023 19253 ts_tablet_manager.cc:1434] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:41.260134 19251 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.260433 19255 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.261584 19251 tablet_bootstrap.cc:654] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.261902 19255 tablet_bootstrap.cc:654] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.266181 19253 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.267922 19253 tablet_bootstrap.cc:654] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.269662 19253 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.269793 19253 ts_tablet_manager.cc:1403] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251023 20:16:41.269866 19251 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.269986 19251 ts_tablet_manager.cc:1403] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20251023 20:16:41.270296 19255 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.270421 19255 ts_tablet_manager.cc:1403] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20251023 20:16:41.270469 19253 raft_consensus.cc:359] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.270515 19251 raft_consensus.cc:359] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.270628 19253 raft_consensus.cc:385] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.270639 19251 raft_consensus.cc:385] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.270702 19253 raft_consensus.cc:740] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.270705 19251 raft_consensus.cc:740] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.270929 19251 consensus_queue.cc:260] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.270918 19253 consensus_queue.cc:260] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.271024 19255 raft_consensus.cc:359] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.271252 19255 raft_consensus.cc:385] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.271306 19255 raft_consensus.cc:740] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.271329 19251 ts_tablet_manager.cc:1434] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:16:41.271549 19251 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.271479 19255 consensus_queue.cc:260] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.271337 19253 ts_tablet_manager.cc:1434] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:16:41.272073 19253 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.272773 19251 tablet_bootstrap.cc:654] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.273326 19253 tablet_bootstrap.cc:654] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.274158 19255 ts_tablet_manager.cc:1434] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251023 20:16:41.274956 19255 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.277522 19255 tablet_bootstrap.cc:654] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.279088 19255 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.279201 19255 ts_tablet_manager.cc:1403] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:16:41.279661 19255 raft_consensus.cc:359] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.279783 19255 raft_consensus.cc:385] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.279844 19255 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.280074 19255 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.280431 19255 ts_tablet_manager.cc:1434] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:41.280645 19255 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.281970 19255 tablet_bootstrap.cc:654] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.283551 19253 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.283591 19251 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.283767 19251 ts_tablet_manager.cc:1403] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.004s
I20251023 20:16:41.283767 19253 ts_tablet_manager.cc:1403] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20251023 20:16:41.283911 19255 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.284060 19255 ts_tablet_manager.cc:1403] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:16:41.284444 19253 raft_consensus.cc:359] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.284423 19251 raft_consensus.cc:359] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.284665 19253 raft_consensus.cc:385] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.284781 19253 raft_consensus.cc:740] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.284696 19251 raft_consensus.cc:385] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.284694 19255 raft_consensus.cc:359] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.284889 19251 raft_consensus.cc:740] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.284912 19255 raft_consensus.cc:385] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.284976 19255 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.285044 19253 consensus_queue.cc:260] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.285044 19251 consensus_queue.cc:260] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.285189 19255 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.285518 19251 ts_tablet_manager.cc:1434] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:16:41.285781 19251 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.286156 19255 ts_tablet_manager.cc:1434] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:41.286413 19255 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.287194 19251 tablet_bootstrap.cc:654] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.287487 19253 ts_tablet_manager.cc:1434] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251023 20:16:41.287740 19255 tablet_bootstrap.cc:654] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.289055 19253 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.292178 19253 tablet_bootstrap.cc:654] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.293967 19255 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.294078 19255 ts_tablet_manager.cc:1403] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20251023 20:16:41.294250 19251 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.294353 19251 ts_tablet_manager.cc:1403] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20251023 20:16:41.294631 19255 raft_consensus.cc:359] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.294754 19255 raft_consensus.cc:385] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.294814 19255 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.294826 19251 raft_consensus.cc:359] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.294977 19251 raft_consensus.cc:385] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.295034 19251 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.294998 19255 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.295209 19251 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.295490 19255 ts_tablet_manager.cc:1434] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:41.295675 19251 ts_tablet_manager.cc:1434] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:41.295711 19255 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.295966 19251 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.297521 19255 tablet_bootstrap.cc:654] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.297770 19251 tablet_bootstrap.cc:654] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.301192 19253 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.301324 19253 ts_tablet_manager.cc:1403] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20251023 20:16:41.302032 19251 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.302173 19251 ts_tablet_manager.cc:1403] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20251023 20:16:41.302052 19253 raft_consensus.cc:359] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.302271 19253 raft_consensus.cc:385] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.302349 19253 raft_consensus.cc:740] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.302443 19255 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.302553 19255 ts_tablet_manager.cc:1403] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251023 20:16:41.302557 19253 consensus_queue.cc:260] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.302770 19251 raft_consensus.cc:359] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.302903 19251 raft_consensus.cc:385] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.302960 19251 raft_consensus.cc:740] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.303110 19255 raft_consensus.cc:359] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.303117 19251 consensus_queue.cc:260] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.303247 19255 raft_consensus.cc:385] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.303300 19255 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.303510 19255 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.303629 19251 ts_tablet_manager.cc:1434] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:16:41.303869 19251 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.304719 19255 ts_tablet_manager.cc:1434] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:16:41.305063 19255 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.305117 19253 ts_tablet_manager.cc:1434] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:16:41.305601 19251 tablet_bootstrap.cc:654] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.306751 19255 tablet_bootstrap.cc:654] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.309098 19253 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.310365 19255 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.310509 19255 ts_tablet_manager.cc:1403] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251023 20:16:41.310988 19255 raft_consensus.cc:359] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.311141 19255 raft_consensus.cc:385] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.311246 19255 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.311305 19253 tablet_bootstrap.cc:654] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.311342 19256 raft_consensus.cc:493] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.311509 19255 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.311461 19256 raft_consensus.cc:515] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.312381 19256 leader_election.cc:290] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:41.313541 19261 raft_consensus.cc:493] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.313680 19261 raft_consensus.cc:515] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.314837 19261 leader_election.cc:290] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.316979 19251 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.317113 19251 ts_tablet_manager.cc:1403] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20251023 20:16:41.317163 19253 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.317294 19253 ts_tablet_manager.cc:1403] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20251023 20:16:41.317759 19251 raft_consensus.cc:359] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.317854 19253 raft_consensus.cc:359] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.317938 19251 raft_consensus.cc:385] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.318020 19251 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.318018 19253 raft_consensus.cc:385] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.318146 19253 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.318249 19251 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.318409 19253 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.318730 19251 ts_tablet_manager.cc:1434] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 20:16:41.318985 19251 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.320749 19251 tablet_bootstrap.cc:654] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.320878 19253 ts_tablet_manager.cc:1434] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:16:41.321964 19253 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.322521 19257 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.322683 19257 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.323753 19257 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.333132 19255 ts_tablet_manager.cc:1434] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20251023 20:16:41.333518 19255 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.335778 19255 tablet_bootstrap.cc:654] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.347724 19255 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.347908 19255 ts_tablet_manager.cc:1403] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20251023 20:16:41.348711 19255 raft_consensus.cc:359] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.348944 19255 raft_consensus.cc:385] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.349021 19255 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.349216 19255 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.349748 19255 ts_tablet_manager.cc:1434] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:41.350032 19255 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.351758 19255 tablet_bootstrap.cc:654] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.357131 19255 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.357288 19255 ts_tablet_manager.cc:1403] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20251023 20:16:41.358000 19255 raft_consensus.cc:359] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.358222 19255 raft_consensus.cc:385] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.358321 19255 raft_consensus.cc:740] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.358573 19255 consensus_queue.cc:260] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.361159 19255 ts_tablet_manager.cc:1434] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:16:41.361502 19253 tablet_bootstrap.cc:654] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.362278 19132 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
W20251023 20:16:41.363102 19170 leader_election.cc:343] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.363438 19255 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.365386 19255 tablet_bootstrap.cc:654] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.367655 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.367985 19206 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.368667 19019 leader_election.cc:304] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.369004 19261 raft_consensus.cc:2804] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.369105 19261 raft_consensus.cc:493] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.369201 19261 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.370179 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.370489 19057 raft_consensus.cc:2468] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.370977 19172 leader_election.cc:304] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters: f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.371379 19256 raft_consensus.cc:2804] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.371479 19256 raft_consensus.cc:493] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.371557 19256 raft_consensus.cc:3060] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.372155 19261 raft_consensus.cc:515] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.373240 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.373404 19206 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.374293 19256 raft_consensus.cc:515] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.375131 19132 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
W20251023 20:16:41.375713 19170 leader_election.cc:343] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.376171 19206 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 1.
I20251023 20:16:41.376382 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.376516 19019 leader_election.cc:304] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.376554 19057 raft_consensus.cc:3060] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.377295 19261 leader_election.cc:290] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.377466 19261 raft_consensus.cc:2804] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.377573 19261 raft_consensus.cc:697] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:41.377816 19261 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.379640 19057 raft_consensus.cc:2468] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.380324 19172 leader_election.cc:304] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters: f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.380805 19256 leader_election.cc:290] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:41.381068 19256 raft_consensus.cc:2804] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.381268 19256 raft_consensus.cc:697] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:41.381650 19256 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.394119 19253 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.394331 19253 ts_tablet_manager.cc:1403] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.072s user 0.003s sys 0.001s
I20251023 20:16:41.395344 19253 raft_consensus.cc:359] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.395622 19253 raft_consensus.cc:385] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.395789 19253 raft_consensus.cc:740] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.396667 19297 raft_consensus.cc:493] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.396878 19297 raft_consensus.cc:515] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.398339 19253 consensus_queue.cc:260] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.399019 19132 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.398977 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.399206 19132 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.399379 19297 leader_election.cc:290] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.399530 19297 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.399586 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37a00c71a3934405917c05725c94dde5" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.399713 19297 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.399830 19130 raft_consensus.cc:2468] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.399960 19251 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.400106 19251 ts_tablet_manager.cc:1403] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.081s user 0.032s sys 0.047s
I20251023 20:16:41.400189 19020 leader_election.cc:304] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.400871 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.400879 19251 raft_consensus.cc:359] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.401139 19251 raft_consensus.cc:385] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.401260 19251 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
W20251023 20:16:41.401427 19020 leader_election.cc:343] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.401485 19251 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.403332 19253 ts_tablet_manager.cc:1434] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20251023 20:16:41.400244 19297 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.403538 19297 raft_consensus.cc:2804] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.403648 19297 raft_consensus.cc:493] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.403759 19297 raft_consensus.cc:3060] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.407498 19297 raft_consensus.cc:515] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.408293 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.408423 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37a00c71a3934405917c05725c94dde5" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.408492 19057 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 0.
I20251023 20:16:41.408556 19130 raft_consensus.cc:3060] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.408998 19098 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.409272 19257 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.409355 19257 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.409436 19257 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.409396 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.409621 19206 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 0.
I20251023 20:16:41.411556 19130 raft_consensus.cc:2468] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 1.
I20251023 20:16:41.411912 19020 leader_election.cc:304] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.412343 19257 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.412799 19255 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.412959 19255 ts_tablet_manager.cc:1403] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.050s user 0.006s sys 0.000s
I20251023 20:16:41.413725 19255 raft_consensus.cc:359] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.413810 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.413905 19255 raft_consensus.cc:385] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.413992 19255 raft_consensus.cc:740] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.414032 19206 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.414192 19255 consensus_queue.cc:260] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.414476 19257 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.414738 19255 ts_tablet_manager.cc:1434] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251023 20:16:41.415032 19255 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.414990 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.415162 19057 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.417992 19206 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 1.
I20251023 20:16:41.418498 19095 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.418835 19257 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.418987 19057 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 1.
I20251023 20:16:41.421278 19253 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.425343 19253 tablet_bootstrap.cc:654] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.426781 19294 raft_consensus.cc:2804] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.427819 19256 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.428118 19255 tablet_bootstrap.cc:654] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.428025 19256 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.429139 19256 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.429174 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
W20251023 20:16:41.429665 19170 leader_election.cc:343] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.429868 18954 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
W20251023 20:16:41.430364 19019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.81.195:37799, user_credentials={real_user=slave}} blocked reactor thread for 53827.6us
I20251023 20:16:41.431294 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.431559 19205 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.431717 19291 raft_consensus.cc:493] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.431293 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37a00c71a3934405917c05725c94dde5" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.431913 19206 raft_consensus.cc:2468] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.431943 19291 raft_consensus.cc:515] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.432381 19019 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters: f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.432997 19297 leader_election.cc:290] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.433081 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.433156 19297 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.433235 19297 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.433323 19297 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
W20251023 20:16:41.433530 19020 leader_election.cc:343] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.433887 19291 leader_election.cc:290] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
W20251023 20:16:41.434180 19172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.81.193:36987, user_credentials={real_user=slave}} blocked reactor thread for 53900.1us
I20251023 20:16:41.434767 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.435691 19251 ts_tablet_manager.cc:1434] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.035s user 0.013s sys 0.022s
I20251023 20:16:41.436089 19251 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.436254 19297 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.436555 19057 raft_consensus.cc:2393] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cea0984de40845da94e6f23513009f02 in current term 1: Already voted for candidate 116b80452edd43bc9d822d31be0c5c38 in this term.
I20251023 20:16:41.437654 19172 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cea0984de40845da94e6f23513009f02; no voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.437697 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.439811 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37a00c71a3934405917c05725c94dde5" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.440764 19294 raft_consensus.cc:697] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:41.440768 19297 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
W20251023 20:16:41.441030 19019 leader_election.cc:343] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.440768 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.441336 19206 raft_consensus.cc:3060] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.441476 19019 leader_election.cc:304] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38; no voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.441177 19298 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
W20251023 20:16:41.442051 19020 leader_election.cc:343] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.442427 19257 raft_consensus.cc:697] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:41.442448 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.442476 19297 raft_consensus.cc:2749] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:16:41.443280 19251 tablet_bootstrap.cc:654] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.443838 19255 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.444283 19255 ts_tablet_manager.cc:1403] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.001s
I20251023 20:16:41.442994 19257 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.441372 18953 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.449697 19255 raft_consensus.cc:359] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.449880 19255 raft_consensus.cc:385] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.449972 19255 raft_consensus.cc:740] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.450187 19255 consensus_queue.cc:260] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.450587 19255 ts_tablet_manager.cc:1434] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251023 20:16:41.450866 19255 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: Bootstrap starting.
I20251023 20:16:41.451365 19253 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.451548 19253 ts_tablet_manager.cc:1403] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.001s
I20251023 20:16:41.444152 19294 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.455698 18953 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 0 to 1, leader changed from <none> to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.456768 19255 tablet_bootstrap.cc:654] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.456568 19253 raft_consensus.cc:359] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.459026 19253 raft_consensus.cc:385] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.459213 19253 raft_consensus.cc:740] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.459818 19206 raft_consensus.cc:2468] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 1.
I20251023 20:16:41.460879 19253 consensus_queue.cc:260] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.461623 19298 raft_consensus.cc:2749] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:16:41.462567 19253 ts_tablet_manager.cc:1434] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.011s user 0.000s sys 0.003s
I20251023 20:16:41.462412 18953 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.463011 19253 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.464527 19253 tablet_bootstrap.cc:654] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.465129 19205 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 1.
I20251023 20:16:41.465678 19019 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters: f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.466090 19294 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.466238 19294 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:41.466503 19294 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.467250 19253 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.467418 19253 ts_tablet_manager.cc:1403] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:16:41.468298 19253 raft_consensus.cc:359] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.468467 19253 raft_consensus.cc:385] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.468551 19253 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.468858 19253 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.469137 19251 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.469302 19251 ts_tablet_manager.cc:1403] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.009s
I20251023 20:16:41.469568 18953 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.470036 19251 raft_consensus.cc:359] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.470230 19251 raft_consensus.cc:385] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.470348 19251 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.470602 19251 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.471410 19253 ts_tablet_manager.cc:1434] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:16:41.471493 19294 raft_consensus.cc:493] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.471622 19294 raft_consensus.cc:515] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.472201 19294 leader_election.cc:290] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.473105 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.473237 19251 ts_tablet_manager.cc:1434] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:16:41.473397 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
W20251023 20:16:41.473618 19019 leader_election.cc:343] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): Illegal state: must be running to vote when last-logged opid is not known
W20251023 20:16:41.473846 19020 leader_election.cc:343] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:16:41.474043 19251 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.473990 19020 leader_election.cc:304] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38; no voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:41.474737 19294 raft_consensus.cc:2749] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:16:41.475184 19253 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.475991 19251 tablet_bootstrap.cc:654] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.476521 19255 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: No bootstrap required, opened a new log
I20251023 20:16:41.476697 19255 ts_tablet_manager.cc:1403] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.004s
I20251023 20:16:41.477195 19253 tablet_bootstrap.cc:654] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.477490 19255 raft_consensus.cc:359] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.477697 19255 raft_consensus.cc:385] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.477799 19255 raft_consensus.cc:740] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.478152 19251 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.478044 19255 consensus_queue.cc:260] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.478358 19251 ts_tablet_manager.cc:1403] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:16:41.479076 19253 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.479075 19251 raft_consensus.cc:359] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.479300 19251 raft_consensus.cc:385] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.479327 19255 ts_tablet_manager.cc:1434] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:16:41.479426 19251 raft_consensus.cc:740] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.479373 19253 ts_tablet_manager.cc:1403] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:16:41.479662 19251 consensus_queue.cc:260] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.480224 19251 ts_tablet_manager.cc:1434] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:41.480228 19253 raft_consensus.cc:359] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.480520 19253 raft_consensus.cc:385] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.480540 19251 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.480649 19253 raft_consensus.cc:740] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.480893 19253 consensus_queue.cc:260] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.481496 19253 ts_tablet_manager.cc:1434] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:16:41.481812 19253 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.482332 19251 tablet_bootstrap.cc:654] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.483556 19253 tablet_bootstrap.cc:654] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.486012 19253 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.486117 19253 ts_tablet_manager.cc:1403] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251023 20:16:41.486598 19253 raft_consensus.cc:359] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.486718 19253 raft_consensus.cc:385] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.486780 19253 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.486941 19253 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.487367 19253 ts_tablet_manager.cc:1434] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:41.487586 19253 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: Bootstrap starting.
I20251023 20:16:41.488916 19253 tablet_bootstrap.cc:654] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.490137 19251 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.490321 19251 ts_tablet_manager.cc:1403] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20251023 20:16:41.490921 19253 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: No bootstrap required, opened a new log
I20251023 20:16:41.491076 19253 ts_tablet_manager.cc:1403] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:16:41.490967 19251 raft_consensus.cc:359] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.491197 19251 raft_consensus.cc:385] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.491357 19251 raft_consensus.cc:740] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.491612 19251 consensus_queue.cc:260] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.491792 19253 raft_consensus.cc:359] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.492007 19253 raft_consensus.cc:385] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.492108 19253 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.492118 19251 ts_tablet_manager.cc:1434] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:16:41.492442 19251 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: Bootstrap starting.
I20251023 20:16:41.492353 19253 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.492926 19253 ts_tablet_manager.cc:1434] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:16:41.493981 19251 tablet_bootstrap.cc:654] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: Neither blocks nor log segments found. Creating new log.
I20251023 20:16:41.495651 19251 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: No bootstrap required, opened a new log
I20251023 20:16:41.495826 19251 ts_tablet_manager.cc:1403] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251023 20:16:41.496542 19251 raft_consensus.cc:359] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.496755 19251 raft_consensus.cc:385] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:16:41.496865 19251 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Initialized, Role: FOLLOWER
I20251023 20:16:41.497087 19251 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.497457 19251 ts_tablet_manager.cc:1434] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:16:41.503240 19298 raft_consensus.cc:493] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.503358 19298 raft_consensus.cc:515] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.503883 19298 leader_election.cc:290] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.504159 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "887ca8402e81417c94da83fdc1b75862" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.504235 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "887ca8402e81417c94da83fdc1b75862" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.504325 19057 raft_consensus.cc:2468] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.504423 19130 raft_consensus.cc:2468] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.504736 19172 leader_election.cc:304] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.505030 19298 raft_consensus.cc:2804] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.505123 19298 raft_consensus.cc:493] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.505187 19298 raft_consensus.cc:3060] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.506978 19298 raft_consensus.cc:515] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.507411 19298 leader_election.cc:290] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.507781 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "887ca8402e81417c94da83fdc1b75862" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.507820 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "887ca8402e81417c94da83fdc1b75862" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.507917 19057 raft_consensus.cc:3060] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.507992 19130 raft_consensus.cc:3060] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.509866 19130 raft_consensus.cc:2468] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.509907 19057 raft_consensus.cc:2468] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.510205 19170 leader_election.cc:304] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.510452 19298 raft_consensus.cc:2804] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.510571 19298 raft_consensus.cc:697] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:41.510823 19298 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.511423 19295 raft_consensus.cc:493] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.511566 19295 raft_consensus.cc:515] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.512331 19295 leader_election.cc:290] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.512620 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.512825 19057 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.512831 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.513068 19130 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.513211 19172 leader_election.cc:304] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.513427 19295 raft_consensus.cc:2804] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.513552 19295 raft_consensus.cc:493] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.513657 19295 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.513532 18953 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
I20251023 20:16:41.516104 19295 raft_consensus.cc:515] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.516817 19295 leader_election.cc:290] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.517002 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.517052 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.517216 19130 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.517246 19057 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.519879 19130 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.519878 19057 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.520314 19170 leader_election.cc:304] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.520617 19295 raft_consensus.cc:2804] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.520747 19295 raft_consensus.cc:697] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:41.521006 19295 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.523280 18953 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
I20251023 20:16:41.533703 19295 raft_consensus.cc:493] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.533864 19295 raft_consensus.cc:515] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.534328 19295 leader_election.cc:290] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.534737 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.534761 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.534914 19130 raft_consensus.cc:2468] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.534921 19057 raft_consensus.cc:2468] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.535269 19170 leader_election.cc:304] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.535517 19295 raft_consensus.cc:2804] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.535612 19295 raft_consensus.cc:493] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.535707 19295 raft_consensus.cc:3060] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.537570 19295 raft_consensus.cc:515] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.538029 19295 leader_election.cc:290] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.538326 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.538415 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.538484 19057 raft_consensus.cc:3060] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.538581 19130 raft_consensus.cc:3060] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.540269 19057 raft_consensus.cc:2468] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.540320 19130 raft_consensus.cc:2468] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.540629 19172 leader_election.cc:304] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.540920 19295 raft_consensus.cc:2804] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.541083 19295 raft_consensus.cc:697] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:41.541352 19295 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.543627 18953 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
I20251023 20:16:41.562152 19257 raft_consensus.cc:493] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.562351 19257 raft_consensus.cc:515] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.562971 19257 leader_election.cc:290] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.563414 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.563411 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.563592 19205 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 0.
I20251023 20:16:41.563711 19057 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 0.
I20251023 20:16:41.563987 19095 leader_election.cc:304] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.564244 19257 raft_consensus.cc:2804] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.564344 19257 raft_consensus.cc:493] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.564445 19257 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.565922 19299 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.566037 19299 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.566380 19257 raft_consensus.cc:515] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.566708 19299 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.566962 19257 leader_election.cc:290] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.567040 19057 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.567095 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.567265 19205 raft_consensus.cc:2393] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f909677d13e94318ad9a3b1b2db05723 in current term 1: Already voted for candidate 116b80452edd43bc9d822d31be0c5c38 in this term.
I20251023 20:16:41.567375 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.567629 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.567639 19098 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f909677d13e94318ad9a3b1b2db05723; no voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02
I20251023 20:16:41.567835 19056 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.567838 19206 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.567950 19257 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.570032 19206 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 1.
I20251023 20:16:41.570678 19095 leader_election.cc:304] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.570778 19257 raft_consensus.cc:2749] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:16:41.570946 19299 raft_consensus.cc:2804] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.571197 19299 raft_consensus.cc:697] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:41.571199 19056 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 1.
I20251023 20:16:41.571506 19299 consensus_queue.cc:237] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.573800 18953 catalog_manager.cc:5649] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 0 to 1, leader changed from <none> to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.629882 19295 raft_consensus.cc:493] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.630089 19295 raft_consensus.cc:515] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.630740 19295 leader_election.cc:290] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.631160 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.631157 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.631402 19056 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.631443 19130 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 0.
I20251023 20:16:41.631796 19172 leader_election.cc:304] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.632061 19295 raft_consensus.cc:2804] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.632167 19295 raft_consensus.cc:493] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.632258 19295 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.633983 19295 raft_consensus.cc:515] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.634490 19295 leader_election.cc:290] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:41.634907 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.634910 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.635068 19130 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.635120 19056 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.636979 19130 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.636961 19056 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 1.
I20251023 20:16:41.637378 19170 leader_election.cc:304] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.637660 19295 raft_consensus.cc:2804] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.637780 19295 raft_consensus.cc:697] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:41.638037 19295 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.640331 18953 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
I20251023 20:16:41.643853 19299 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.643976 19299 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.644480 19299 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.644932 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.644932 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.645100 19206 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 0.
I20251023 20:16:41.645146 19056 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 0.
I20251023 20:16:41.645457 19095 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.645704 19299 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.645816 19299 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.645903 19299 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.647617 19299 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.648077 19299 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.648412 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:41.648454 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.648574 19056 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.648617 19206 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.650432 19056 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 1.
I20251023 20:16:41.650528 19206 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 1.
I20251023 20:16:41.650782 19098 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:41.651044 19299 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.651149 19299 raft_consensus.cc:697] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:41.651408 19299 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.653558 18953 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 0 to 1, leader changed from <none> to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.769358 19294 raft_consensus.cc:493] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.769649 19294 raft_consensus.cc:515] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.770294 19294 leader_election.cc:290] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.770753 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.770776 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723" is_pre_election: true
I20251023 20:16:41.770957 19206 raft_consensus.cc:2468] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.770965 19130 raft_consensus.cc:2468] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 0.
I20251023 20:16:41.771343 19019 leader_election.cc:304] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.771577 19294 raft_consensus.cc:2804] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:16:41.771669 19294 raft_consensus.cc:493] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:16:41.771733 19294 raft_consensus.cc:3060] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.774983 19294 raft_consensus.cc:515] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.775522 19294 leader_election.cc:290] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.775879 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:41.775879 19130 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:41.776117 19130 raft_consensus.cc:3060] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.776114 19206 raft_consensus.cc:3060] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.778672 19206 raft_consensus.cc:2468] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 1.
I20251023 20:16:41.778678 19130 raft_consensus.cc:2468] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 1.
I20251023 20:16:41.779088 19019 leader_election.cc:304] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:41.779347 19294 raft_consensus.cc:2804] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:16:41.779474 19294 raft_consensus.cc:697] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:41.779723 19294 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.782048 18953 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:41.821278 19130 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.825048 19294 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:41.825048 19299 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:41.831082 19299 raft_consensus.cc:493] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:16:41.831261 19299 raft_consensus.cc:515] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:41.832244 19299 leader_election.cc:290] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:41.837958 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116b80452edd43bc9d822d31be0c5c38" is_pre_election: true
I20251023 20:16:41.843942 19056 raft_consensus.cc:2393] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f909677d13e94318ad9a3b1b2db05723 in current term 1: Already voted for candidate cea0984de40845da94e6f23513009f02 in this term.
I20251023 20:16:41.858479 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:41.867758 19295 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20251023 20:16:41.869127 19206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea0984de40845da94e6f23513009f02" is_pre_election: true
I20251023 20:16:41.872227 19294 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20251023 20:16:41.876802 19095 leader_election.cc:304] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f909677d13e94318ad9a3b1b2db05723; no voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02
I20251023 20:16:41.877517 19261 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20251023 20:16:41.878963 19299 raft_consensus.cc:3060] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:16:41.904058 19297 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20251023 20:16:41.904976 19261 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20251023 20:16:41.916121 19299 raft_consensus.cc:2749] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:16:41.930864 19295 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20251023 20:16:41.933945 19257 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20251023 20:16:41.984184 19257 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.004652 19298 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20251023 20:16:42.012816 19298 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251023 20:16:42.027071 19261 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
W20251023 20:16:42.050583 19172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.81.193:36987, user_credentials={real_user=slave}} blocked reactor thread for 45437.1us
I20251023 20:16:42.055508 19298 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.085683 19332 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.086414 19298 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.076953 19206 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:53226
I20251023 20:16:42.087285 19206 raft_consensus.cc:606] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:42.090461 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:16:42.092808 19346 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.099447 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:42.109335 19298 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.145905 19298 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.147729 19298 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.153650 19295 raft_consensus.cc:993] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:42.153786 19295 raft_consensus.cc:1081] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:42.154443 19132 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36232
I20251023 20:16:42.154592 19132 raft_consensus.cc:493] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:42.154667 19132 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:42.157861 19132 raft_consensus.cc:515] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:42.158468 19132 leader_election.cc:290] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 2 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:42.158782 19130 raft_consensus.cc:1240] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Rejecting Update request from peer cea0984de40845da94e6f23513009f02 for earlier term 1. Current term is 2. Ops: []
I20251023 20:16:42.161454 19204 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
W20251023 20:16:42.161582 19186 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53230: Service unavailable: leader transfer in progress
I20251023 20:16:42.161677 19204 raft_consensus.cc:3055] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:42.161741 19204 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:42.161939 19204 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:42.162236 19204 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:42.162873 19297 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:16:42.163296 19186 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53230: Illegal state: replica cea0984de40845da94e6f23513009f02 is not leader of this config: current role FOLLOWER
I20251023 20:16:42.168509 19204 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 2.
W20251023 20:16:42.161476 19179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53230: Service unavailable: leader transfer in progress
I20251023 20:16:42.170019 19095 leader_election.cc:304] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:42.171686 19056 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:16:42.171809 19257 raft_consensus.cc:2804] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:42.171914 19257 raft_consensus.cc:697] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:42.172164 19257 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:42.177497 19299 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20251023 20:16:42.197893 19178 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53230: Illegal state: replica cea0984de40845da94e6f23513009f02 is not leader of this config: current role FOLLOWER
I20251023 20:16:42.167625 19054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:42.227725 19206 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:16:42.228264 19337 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:16:42.226799 19301 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20251023 20:16:42.180078 18947 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 1 to 2, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:42.289407 19037 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55954: Illegal state: replica 116b80452edd43bc9d822d31be0c5c38 is not leader of this config: current role FOLLOWER
W20251023 20:16:42.297330 19034 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55954: Illegal state: replica 116b80452edd43bc9d822d31be0c5c38 is not leader of this config: current role FOLLOWER
I20251023 20:16:42.317953 19205 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:16:42.318481 19383 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:16:42.317984 19057 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
W20251023 20:16:42.300191 19036 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55954: Illegal state: replica 116b80452edd43bc9d822d31be0c5c38 is not leader of this config: current role FOLLOWER
W20251023 20:16:42.298415 19033 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55954: Illegal state: replica 116b80452edd43bc9d822d31be0c5c38 is not leader of this config: current role FOLLOWER
I20251023 20:16:42.337951 19057 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20251023 20:16:42.345638 19342 log.cc:927] Time spent T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Append to log took a long time: real 0.104s user 0.000s sys 0.000s
I20251023 20:16:42.338717 19337 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.013s
I20251023 20:16:43.434266 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:43.437076 19404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:43.436650 19403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:43.441694 19406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:43.442461 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:43.442662 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:43.442711 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:43.442751 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250603442748 us; error 0 us; skew 500 ppm
I20251023 20:16:43.442909 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:43.444252 15687 webserver.cc:492] Webserver started at http://127.15.81.196:36577/ using document root <none> and password file <none>
I20251023 20:16:43.444479 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:43.444559 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:43.444682 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:43.445333 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
format_stamp: "Formatted at 2025-10-23 20:16:43 on dist-test-slave-jr94"
I20251023 20:16:43.447660 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 20:16:43.449188 19411 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.449576 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251023 20:16:43.449666 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
format_stamp: "Formatted at 2025-10-23 20:16:43 on dist-test-slave-jr94"
I20251023 20:16:43.449781 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:43.467563 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:43.468088 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:43.468595 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:43.469507 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:43.469559 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.469627 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:43.469669 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.485746 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:34839
I20251023 20:16:43.485783 19473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:34839 every 8 connection(s)
I20251023 20:16:43.487545 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:43.490636 19478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:43.491050 19479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:43.492338 19482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:43.493692 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:43.493912 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:43.493963 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:43.494000 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250603493996 us; error 0 us; skew 500 ppm
I20251023 20:16:43.494138 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:43.495236 15687 webserver.cc:492] Webserver started at http://127.15.81.197:40777/ using document root <none> and password file <none>
I20251023 20:16:43.495432 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:43.495496 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:43.495613 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:43.496268 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-4-root/instance:
uuid: "3c36454265a141c0adf91706fa3115e9"
format_stamp: "Formatted at 2025-10-23 20:16:43 on dist-test-slave-jr94"
I20251023 20:16:43.498093 19474 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45023
I20251023 20:16:43.498210 19474 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:43.498662 19474 heartbeater.cc:507] Master 127.15.81.254:45023 requested a full tablet report, sending...
I20251023 20:16:43.499145 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251023 20:16:43.499521 18947 ts_manager.cc:194] Registered new tserver with Master: 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:16:43.500372 18947 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35030
I20251023 20:16:43.500810 19486 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.501379 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:43.501492 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-4-root
uuid: "3c36454265a141c0adf91706fa3115e9"
format_stamp: "Formatted at 2025-10-23 20:16:43 on dist-test-slave-jr94"
I20251023 20:16:43.501598 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-4-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-4-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-4-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:43.528923 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:43.529413 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:43.529883 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:43.530584 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:43.530633 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.530689 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:43.530730 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.542672 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.197:33571
I20251023 20:16:43.542703 19549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.197:33571 every 8 connection(s)
I20251023 20:16:43.544256 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:16:43.546804 19553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:43.547075 19554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:16:43.548226 19556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:16:43.550077 19550 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45023
I20251023 20:16:43.550181 19550 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:43.550523 19550 heartbeater.cc:507] Master 127.15.81.254:45023 requested a full tablet report, sending...
I20251023 20:16:43.551287 18947 ts_manager.cc:194] Registered new tserver with Master: 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571)
I20251023 20:16:43.551915 18947 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35032
I20251023 20:16:43.553524 15687 server_base.cc:1047] running on GCE node
I20251023 20:16:43.553736 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:16:43.553786 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:16:43.553807 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250603553805 us; error 0 us; skew 500 ppm
I20251023 20:16:43.553900 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:16:43.554980 15687 webserver.cc:492] Webserver started at http://127.15.81.198:45919/ using document root <none> and password file <none>
I20251023 20:16:43.555146 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:16:43.555199 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:16:43.555282 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:16:43.555831 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-5-root/instance:
uuid: "329b90f727b745ddacc774c89c3a4428"
format_stamp: "Formatted at 2025-10-23 20:16:43 on dist-test-slave-jr94"
I20251023 20:16:43.558063 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:43.559518 19561 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.559837 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:43.559931 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-5-root
uuid: "329b90f727b745ddacc774c89c3a4428"
format_stamp: "Formatted at 2025-10-23 20:16:43 on dist-test-slave-jr94"
I20251023 20:16:43.560024 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-5-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-5-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-5-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:16:43.586941 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:16:43.587442 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:16:43.587962 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:16:43.588709 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:16:43.588764 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.588862 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:16:43.588917 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:16:43.602792 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.198:43569
I20251023 20:16:43.602854 19623 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.198:43569 every 8 connection(s)
I20251023 20:16:43.609122 19624 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45023
I20251023 20:16:43.609239 19624 heartbeater.cc:461] Registering TS with master...
I20251023 20:16:43.609562 19624 heartbeater.cc:507] Master 127.15.81.254:45023 requested a full tablet report, sending...
I20251023 20:16:43.610275 18947 ts_manager.cc:194] Registered new tserver with Master: 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:16:43.610947 18947 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35036
I20251023 20:16:43.954960 19054 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:43.957384 19203 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:43.957746 19128 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:43.957958 19261 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:43.958352 19630 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.001s
I20251023 20:16:43.960136 19261 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:43.961539 19128 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:43.963088 19203 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:43.965642 18947 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 29, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:43.966759 19020 consensus_peers.cc:597] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37a00c71a3934405917c05725c94dde5. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:43.975086 19203 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:43.977123 19128 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:43.977730 19054 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:43.978111 19638 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:43.978451 19401 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:43.980896 19401 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:43.984630 19054 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:43.984727 19128 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:43.985529 19056 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:43.987475 19203 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:43.987954 19630 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:43.988131 19131 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:43.988884 19630 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:43.988801 18952 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index -1 to 28, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:43.990083 19172 consensus_peers.cc:597] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 -> Peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571): Couldn't send request to peer 3c36454265a141c0adf91706fa3115e9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a9aada4eefa459ea9801d5506a2b82e. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:43.990672 19630 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:43.991163 19203 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:43.994146 19131 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:43.997134 18951 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 29, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:43.999643 19203 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:44.001634 19054 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:44.002228 19641 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:44.002681 19131 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:44.003839 19641 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:44.004595 19401 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:44.006922 19131 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
W20251023 20:16:44.007644 19170 consensus_peers.cc:597] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 887ca8402e81417c94da83fdc1b75862. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:44.008952 19054 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:44.013273 18951 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
W20251023 20:16:44.014451 19020 consensus_peers.cc:597] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 -> Peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839): Couldn't send request to peer 4cccadfc20fb475c9e6c1ce6cd3ba850. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4ff9ca441b14a468709c0efd0ae984e. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:44.021982 19131 consensus_queue.cc:237] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:44.024109 19203 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:44.024398 19054 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:44.024647 19645 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:44.024964 19337 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:44.026793 19337 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:44.027545 19054 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
W20251023 20:16:44.030375 19098 consensus_peers.cc:597] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 -> Peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571): Couldn't send request to peer 3c36454265a141c0adf91706fa3115e9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 508756851fc14f93a7d310b11beb15cb. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:44.032557 18947 catalog_manager.cc:5649] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New cstate: current_term: 1 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:44.040939 19203 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:44.042523 19202 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:44.044530 19131 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:44.045306 19641 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:44.046739 19054 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:44.047246 19643 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:44.047194 19638 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:44.047842 19131 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:44.048789 19056 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:44.053206 18952 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:44.053598 19170 consensus_peers.cc:597] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 -> Peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839): Couldn't send request to peer 4cccadfc20fb475c9e6c1ce6cd3ba850. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 461a7ab7c21c40fbb4ef24809dad5831. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:44.099798 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:44.102566 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:44.102756 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:44.484697 19651 ts_tablet_manager.cc:933] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Initiating tablet copy from peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:44.485352 19651 tablet_copy_client.cc:323] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:37799
I20251023 20:16:44.491595 19216 tablet_copy_service.cc:140] P cea0984de40845da94e6f23513009f02: Received BeginTabletCopySession request for tablet 2a9aada4eefa459ea9801d5506a2b82e from peer 3c36454265a141c0adf91706fa3115e9 ({username='slave'} at 127.0.0.1:53250)
I20251023 20:16:44.491715 19216 tablet_copy_service.cc:161] P cea0984de40845da94e6f23513009f02: Beginning new tablet copy session on tablet 2a9aada4eefa459ea9801d5506a2b82e from peer 3c36454265a141c0adf91706fa3115e9 at {username='slave'} at 127.0.0.1:53250: session id = 3c36454265a141c0adf91706fa3115e9-2a9aada4eefa459ea9801d5506a2b82e
I20251023 20:16:44.493872 19216 tablet_copy_source_session.cc:215] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:44.495028 19651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9aada4eefa459ea9801d5506a2b82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:44.500356 19651 tablet_copy_client.cc:806] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:44.500653 19651 tablet_copy_client.cc:670] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:44.501791 19474 heartbeater.cc:499] Master 127.15.81.254:45023 was elected leader, sending a full tablet report...
I20251023 20:16:44.502039 19652 tablet_copy_client.cc:1025] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:44.511195 19655 ts_tablet_manager.cc:933] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Initiating tablet copy from peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:44.511744 19655 tablet_copy_client.cc:323] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:36987
I20251023 20:16:44.518107 19067 tablet_copy_service.cc:140] P 116b80452edd43bc9d822d31be0c5c38: Received BeginTabletCopySession request for tablet f4ff9ca441b14a468709c0efd0ae984e from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 ({username='slave'} at 127.0.0.1:55958)
I20251023 20:16:44.518249 19067 tablet_copy_service.cc:161] P 116b80452edd43bc9d822d31be0c5c38: Beginning new tablet copy session on tablet f4ff9ca441b14a468709c0efd0ae984e from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 at {username='slave'} at 127.0.0.1:55958: session id = 4cccadfc20fb475c9e6c1ce6cd3ba850-f4ff9ca441b14a468709c0efd0ae984e
I20251023 20:16:44.520437 19067 tablet_copy_source_session.cc:215] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:44.521564 19655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ff9ca441b14a468709c0efd0ae984e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:44.526085 19655 tablet_copy_client.cc:806] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:44.526343 19655 tablet_copy_client.cc:670] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:44.527590 19656 tablet_copy_client.cc:1025] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:44.552874 19550 heartbeater.cc:499] Master 127.15.81.254:45023 was elected leader, sending a full tablet report...
I20251023 20:16:44.573849 19659 ts_tablet_manager.cc:933] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Initiating tablet copy from peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:44.574431 19659 tablet_copy_client.cc:323] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:36987
I20251023 20:16:44.580224 19067 tablet_copy_service.cc:140] P 116b80452edd43bc9d822d31be0c5c38: Received BeginTabletCopySession request for tablet 37a00c71a3934405917c05725c94dde5 from peer 329b90f727b745ddacc774c89c3a4428 ({username='slave'} at 127.0.0.1:49630)
I20251023 20:16:44.580341 19067 tablet_copy_service.cc:161] P 116b80452edd43bc9d822d31be0c5c38: Beginning new tablet copy session on tablet 37a00c71a3934405917c05725c94dde5 from peer 329b90f727b745ddacc774c89c3a4428 at {username='slave'} at 127.0.0.1:49630: session id = 329b90f727b745ddacc774c89c3a4428-37a00c71a3934405917c05725c94dde5
I20251023 20:16:44.582454 19067 tablet_copy_source_session.cc:215] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:44.583536 19659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a00c71a3934405917c05725c94dde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:44.588022 19659 tablet_copy_client.cc:806] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:44.588263 19659 tablet_copy_client.cc:670] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:44.589682 19660 tablet_copy_client.cc:1025] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:44.591391 19662 ts_tablet_manager.cc:933] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Initiating tablet copy from peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:44.592104 19662 tablet_copy_client.cc:323] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:37799
I20251023 20:16:44.597931 19216 tablet_copy_service.cc:140] P cea0984de40845da94e6f23513009f02: Received BeginTabletCopySession request for tablet 461a7ab7c21c40fbb4ef24809dad5831 from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 ({username='slave'} at 127.0.0.1:58466)
I20251023 20:16:44.598063 19216 tablet_copy_service.cc:161] P cea0984de40845da94e6f23513009f02: Beginning new tablet copy session on tablet 461a7ab7c21c40fbb4ef24809dad5831 from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 at {username='slave'} at 127.0.0.1:58466: session id = 4cccadfc20fb475c9e6c1ce6cd3ba850-461a7ab7c21c40fbb4ef24809dad5831
I20251023 20:16:44.600291 19216 tablet_copy_source_session.cc:215] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:44.601948 19662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461a7ab7c21c40fbb4ef24809dad5831. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:44.605180 19664 ts_tablet_manager.cc:933] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: Initiating tablet copy from peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:44.606312 19664 tablet_copy_client.cc:323] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:33259
I20251023 20:16:44.608836 19662 tablet_copy_client.cc:806] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:44.609104 19662 tablet_copy_client.cc:670] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:44.610476 19663 tablet_copy_client.cc:1025] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:44.611918 19624 heartbeater.cc:499] Master 127.15.81.254:45023 was elected leader, sending a full tablet report...
I20251023 20:16:44.613480 19142 tablet_copy_service.cc:140] P f909677d13e94318ad9a3b1b2db05723: Received BeginTabletCopySession request for tablet 508756851fc14f93a7d310b11beb15cb from peer 3c36454265a141c0adf91706fa3115e9 ({username='slave'} at 127.0.0.1:49234)
I20251023 20:16:44.613670 19142 tablet_copy_service.cc:161] P f909677d13e94318ad9a3b1b2db05723: Beginning new tablet copy session on tablet 508756851fc14f93a7d310b11beb15cb from peer 3c36454265a141c0adf91706fa3115e9 at {username='slave'} at 127.0.0.1:49234: session id = 3c36454265a141c0adf91706fa3115e9-508756851fc14f93a7d310b11beb15cb
I20251023 20:16:44.617197 19667 ts_tablet_manager.cc:933] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: Initiating tablet copy from peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:44.617403 19142 tablet_copy_source_session.cc:215] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:44.618209 19667 tablet_copy_client.cc:323] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:37799
I20251023 20:16:44.618809 19664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 508756851fc14f93a7d310b11beb15cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:44.624011 19664 tablet_copy_client.cc:806] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:44.624125 19216 tablet_copy_service.cc:140] P cea0984de40845da94e6f23513009f02: Received BeginTabletCopySession request for tablet 887ca8402e81417c94da83fdc1b75862 from peer 329b90f727b745ddacc774c89c3a4428 ({username='slave'} at 127.0.0.1:58470)
I20251023 20:16:44.624240 19664 tablet_copy_client.cc:670] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:44.624248 19216 tablet_copy_service.cc:161] P cea0984de40845da94e6f23513009f02: Beginning new tablet copy session on tablet 887ca8402e81417c94da83fdc1b75862 from peer 329b90f727b745ddacc774c89c3a4428 at {username='slave'} at 127.0.0.1:58470: session id = 329b90f727b745ddacc774c89c3a4428-887ca8402e81417c94da83fdc1b75862
I20251023 20:16:44.625713 19665 tablet_copy_client.cc:1025] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:44.626559 19216 tablet_copy_source_session.cc:215] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:44.627506 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887ca8402e81417c94da83fdc1b75862. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:44.632055 19667 tablet_copy_client.cc:806] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:44.632270 19667 tablet_copy_client.cc:670] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:44.633553 19668 tablet_copy_client.cc:1025] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:46.103116 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:46.105449 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:16:46.105654 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:46.502440 19651 tablet_copy_client.cc:538] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:46.505429 19651 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Bootstrap starting.
I20251023 20:16:46.527915 19655 tablet_copy_client.cc:538] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:46.531188 19655 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap starting.
I20251023 20:16:46.546855 19651 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=110 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:46.547199 19651 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Bootstrap complete.
I20251023 20:16:46.547444 19651 ts_tablet_manager.cc:1403] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Time spent bootstrapping tablet: real 0.042s user 0.036s sys 0.004s
I20251023 20:16:46.548118 19651 raft_consensus.cc:359] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:46.548363 19651 raft_consensus.cc:740] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Initialized, Role: LEARNER
I20251023 20:16:46.548594 19651 consensus_queue.cc:260] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:46.561311 19651 ts_tablet_manager.cc:1434] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Time spent starting tablet: real 0.014s user 0.004s sys 0.012s
I20251023 20:16:46.562090 19216 tablet_copy_service.cc:342] P cea0984de40845da94e6f23513009f02: Request end of tablet copy session 3c36454265a141c0adf91706fa3115e9-2a9aada4eefa459ea9801d5506a2b82e received from {username='slave'} at 127.0.0.1:53250
I20251023 20:16:46.562204 19216 tablet_copy_service.cc:434] P cea0984de40845da94e6f23513009f02: ending tablet copy session 3c36454265a141c0adf91706fa3115e9-2a9aada4eefa459ea9801d5506a2b82e on tablet 2a9aada4eefa459ea9801d5506a2b82e with peer 3c36454265a141c0adf91706fa3115e9
I20251023 20:16:46.564360 19525 raft_consensus.cc:1217] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.28] Dedup: 1.28->[]
I20251023 20:16:46.581650 19655 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:46.582013 19655 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap complete.
I20251023 20:16:46.582289 19655 ts_tablet_manager.cc:1403] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent bootstrapping tablet: real 0.051s user 0.036s sys 0.014s
I20251023 20:16:46.582928 19655 raft_consensus.cc:359] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:46.583112 19655 raft_consensus.cc:740] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Initialized, Role: LEARNER
I20251023 20:16:46.583321 19655 consensus_queue.cc:260] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:46.583986 19655 ts_tablet_manager.cc:1434] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 20:16:46.584560 19067 tablet_copy_service.cc:342] P 116b80452edd43bc9d822d31be0c5c38: Request end of tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-f4ff9ca441b14a468709c0efd0ae984e received from {username='slave'} at 127.0.0.1:55958
I20251023 20:16:46.584652 19067 tablet_copy_service.cc:434] P 116b80452edd43bc9d822d31be0c5c38: ending tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-f4ff9ca441b14a468709c0efd0ae984e on tablet f4ff9ca441b14a468709c0efd0ae984e with peer 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:46.589972 19659 tablet_copy_client.cc:538] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:46.593096 19659 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Bootstrap starting.
I20251023 20:16:46.610767 19662 tablet_copy_client.cc:538] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:46.613902 19662 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap starting.
I20251023 20:16:46.626010 19664 tablet_copy_client.cc:538] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:46.629356 19664 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: Bootstrap starting.
I20251023 20:16:46.634194 19667 tablet_copy_client.cc:538] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:46.637419 19667 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: Bootstrap starting.
I20251023 20:16:46.640648 19659 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=120 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:46.641069 19659 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Bootstrap complete.
I20251023 20:16:46.641374 19659 ts_tablet_manager.cc:1403] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Time spent bootstrapping tablet: real 0.048s user 0.039s sys 0.005s
I20251023 20:16:46.642122 19659 raft_consensus.cc:359] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:46.642417 19659 raft_consensus.cc:740] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Initialized, Role: LEARNER
I20251023 20:16:46.642683 19659 consensus_queue.cc:260] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:46.649765 19659 ts_tablet_manager.cc:1434] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20251023 20:16:46.651114 19067 tablet_copy_service.cc:342] P 116b80452edd43bc9d822d31be0c5c38: Request end of tablet copy session 329b90f727b745ddacc774c89c3a4428-37a00c71a3934405917c05725c94dde5 received from {username='slave'} at 127.0.0.1:49630
I20251023 20:16:46.651238 19067 tablet_copy_service.cc:434] P 116b80452edd43bc9d822d31be0c5c38: ending tablet copy session 329b90f727b745ddacc774c89c3a4428-37a00c71a3934405917c05725c94dde5 on tablet 37a00c71a3934405917c05725c94dde5 with peer 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:46.659916 19662 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=95 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:46.660316 19662 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap complete.
I20251023 20:16:46.660578 19662 ts_tablet_manager.cc:1403] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent bootstrapping tablet: real 0.047s user 0.041s sys 0.000s
I20251023 20:16:46.661365 19662 raft_consensus.cc:359] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:46.661655 19662 raft_consensus.cc:740] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Initialized, Role: LEARNER
I20251023 20:16:46.661881 19662 consensus_queue.cc:260] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:46.662472 19662 ts_tablet_manager.cc:1434] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:16:46.663080 19216 tablet_copy_service.cc:342] P cea0984de40845da94e6f23513009f02: Request end of tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-461a7ab7c21c40fbb4ef24809dad5831 received from {username='slave'} at 127.0.0.1:58466
I20251023 20:16:46.663180 19216 tablet_copy_service.cc:434] P cea0984de40845da94e6f23513009f02: ending tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-461a7ab7c21c40fbb4ef24809dad5831 on tablet 461a7ab7c21c40fbb4ef24809dad5831 with peer 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:46.685671 19667 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=127 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:46.686093 19667 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: Bootstrap complete.
I20251023 20:16:46.686344 19667 ts_tablet_manager.cc:1403] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: Time spent bootstrapping tablet: real 0.049s user 0.027s sys 0.014s
I20251023 20:16:46.686959 19667 raft_consensus.cc:359] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:46.687129 19667 raft_consensus.cc:740] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Initialized, Role: LEARNER
I20251023 20:16:46.687316 19667 consensus_queue.cc:260] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:46.687824 19667 ts_tablet_manager.cc:1434] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 20:16:46.688340 19216 tablet_copy_service.cc:342] P cea0984de40845da94e6f23513009f02: Request end of tablet copy session 329b90f727b745ddacc774c89c3a4428-887ca8402e81417c94da83fdc1b75862 received from {username='slave'} at 127.0.0.1:58470
I20251023 20:16:46.688441 19216 tablet_copy_service.cc:434] P cea0984de40845da94e6f23513009f02: ending tablet copy session 329b90f727b745ddacc774c89c3a4428-887ca8402e81417c94da83fdc1b75862 on tablet 887ca8402e81417c94da83fdc1b75862 with peer 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:46.689826 19664 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=116 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:46.690227 19664 tablet_bootstrap.cc:492] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: Bootstrap complete.
I20251023 20:16:46.690450 19664 ts_tablet_manager.cc:1403] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: Time spent bootstrapping tablet: real 0.061s user 0.034s sys 0.015s
I20251023 20:16:46.691000 19664 raft_consensus.cc:359] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:46.691192 19664 raft_consensus.cc:740] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Initialized, Role: LEARNER
I20251023 20:16:46.691375 19664 consensus_queue.cc:260] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:46.691795 19664 ts_tablet_manager.cc:1434] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:46.692344 19142 tablet_copy_service.cc:342] P f909677d13e94318ad9a3b1b2db05723: Request end of tablet copy session 3c36454265a141c0adf91706fa3115e9-508756851fc14f93a7d310b11beb15cb received from {username='slave'} at 127.0.0.1:49234
I20251023 20:16:46.692447 19142 tablet_copy_service.cc:434] P f909677d13e94318ad9a3b1b2db05723: ending tablet copy session 3c36454265a141c0adf91706fa3115e9-508756851fc14f93a7d310b11beb15cb on tablet 508756851fc14f93a7d310b11beb15cb with peer 3c36454265a141c0adf91706fa3115e9
I20251023 20:16:46.894378 19449 raft_consensus.cc:1217] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.28] Dedup: 1.28->[]
I20251023 20:16:46.921793 19599 raft_consensus.cc:1217] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 20:16:46.935910 19449 raft_consensus.cc:1217] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 20:16:46.991257 19599 raft_consensus.cc:1217] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.28] Dedup: 1.28->[]
I20251023 20:16:47.145063 19525 raft_consensus.cc:1217] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251023 20:16:47.181242 19685 raft_consensus.cc:1064] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 3c36454265a141c0adf91706fa3115e9 to VOTER
I20251023 20:16:47.181953 19685 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:47.183858 19525 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.184306 19131 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.184751 19056 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.184391 19687 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.185146 19672 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.185861 19672 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.189826 19687 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.190250 19525 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.191304 19131 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.196511 18951 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 28 to 29, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.199811 19056 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.209043 19202 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:47.210659 19525 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.210865 19131 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.211119 19685 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.211318 19689 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.213629 19685 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.214104 19131 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.217234 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 2a9aada4eefa459ea9801d5506a2b82e with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:47.217443 19525 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.219501 18952 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.223193 19041 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a9aada4eefa459ea9801d5506a2b82e with delete_type TABLET_DATA_TOMBSTONED (TS 116b80452edd43bc9d822d31be0c5c38 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:47.223827 19697 tablet_replica.cc:333] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:16:47.224071 19697 raft_consensus.cc:2243] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:47.224349 19697 raft_consensus.cc:2272] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:47.225111 19697 ts_tablet_manager.cc:1916] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:47.229846 19697 ts_tablet_manager.cc:1929] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.29
I20251023 20:16:47.229947 19697 log.cc:1199] T 2a9aada4eefa459ea9801d5506a2b82e P 116b80452edd43bc9d822d31be0c5c38: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/2a9aada4eefa459ea9801d5506a2b82e
I20251023 20:16:47.230544 18941 catalog_manager.cc:4985] TS 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): tablet 2a9aada4eefa459ea9801d5506a2b82e (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:47.419610 19687 raft_consensus.cc:1064] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 to VOTER
I20251023 20:16:47.420470 19687 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:47.422791 19449 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.423115 19056 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.423175 19128 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.423290 19689 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.423573 19687 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.423800 19672 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.427183 19672 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.427768 19128 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.431519 19449 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.432032 19056 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.436028 18947 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.437830 19703 raft_consensus.cc:1064] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: attempting to promote NON_VOTER 329b90f727b745ddacc774c89c3a4428 to VOTER
I20251023 20:16:47.438548 19703 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:47.442453 19202 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.442512 19599 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.442508 19128 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.443372 19702 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.443629 19684 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.443885 19203 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:47.444083 19702 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.446179 19131 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.447449 19689 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.449595 19449 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.450081 19672 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.453604 19449 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.452906 19672 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.458328 19684 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.462574 19598 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.463071 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 461a7ab7c21c40fbb4ef24809dad5831 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:47.463467 19131 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.463501 18951 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 29 to 30, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:47.466989 19041 tablet_service.cc:1552] Processing DeleteTablet for tablet 461a7ab7c21c40fbb4ef24809dad5831 with delete_type TABLET_DATA_TOMBSTONED (TS 116b80452edd43bc9d822d31be0c5c38 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:47.467631 19697 tablet_replica.cc:333] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:16:47.467785 19697 raft_consensus.cc:2243] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:47.467929 19697 raft_consensus.cc:2272] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:47.467978 19128 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.469319 18951 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 29 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.471904 19697 ts_tablet_manager.cc:1916] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:47.472613 19203 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.478843 19697 ts_tablet_manager.cc:1929] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.29
I20251023 20:16:47.479120 19697 log.cc:1199] T 461a7ab7c21c40fbb4ef24809dad5831 P 116b80452edd43bc9d822d31be0c5c38: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/461a7ab7c21c40fbb4ef24809dad5831
I20251023 20:16:47.479408 19056 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:47.480289 18941 catalog_manager.cc:4985] TS 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): tablet 461a7ab7c21c40fbb4ef24809dad5831 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:47.480904 19598 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:47.480986 19203 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:47.481352 19703 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:47.481634 19705 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:47.484508 19703 raft_consensus.cc:1064] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: attempting to promote NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 to VOTER
I20251023 20:16:47.484467 19702 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.485322 19703 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:47.486707 19598 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.488243 19449 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.488488 18941 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 37a00c71a3934405917c05725c94dde5 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:47.489367 19203 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.489639 19684 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.490621 19702 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.490778 18952 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index 30 to 31, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:47.491108 19203 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.494870 19113 tablet_service.cc:1552] Processing DeleteTablet for tablet 37a00c71a3934405917c05725c94dde5 with delete_type TABLET_DATA_TOMBSTONED (TS f909677d13e94318ad9a3b1b2db05723 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:36224
I20251023 20:16:47.504983 19128 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.505957 19684 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.509625 19713 tablet_replica.cc:333] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:16:47.509792 19713 raft_consensus.cc:2243] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:47.509954 19713 raft_consensus.cc:2272] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:47.510756 19713 ts_tablet_manager.cc:1916] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:47.512280 19684 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.516348 19202 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.516901 18952 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index 29 to 30, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:47.518056 19713 ts_tablet_manager.cc:1929] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251023 20:16:47.518144 19713 log.cc:1199] T 37a00c71a3934405917c05725c94dde5 P f909677d13e94318ad9a3b1b2db05723: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/37a00c71a3934405917c05725c94dde5
I20251023 20:16:47.518445 19449 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.519186 18939 catalog_manager.cc:4985] TS f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): tablet 37a00c71a3934405917c05725c94dde5 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:47.519340 19128 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.525970 19056 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:47.527352 19449 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:47.527961 19131 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:47.527927 19709 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:47.528401 19702 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:47.530498 19711 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.530941 19449 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.531831 19128 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.534924 18941 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet f4ff9ca441b14a468709c0efd0ae984e with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:47.535492 18946 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:47.538487 19196 tablet_service.cc:1552] Processing DeleteTablet for tablet f4ff9ca441b14a468709c0efd0ae984e with delete_type TABLET_DATA_TOMBSTONED (TS cea0984de40845da94e6f23513009f02 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:47.539160 19716 tablet_replica.cc:333] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:16:47.539393 19716 raft_consensus.cc:2243] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:47.539743 19716 raft_consensus.cc:2272] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:47.540658 19716 ts_tablet_manager.cc:1916] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:47.545624 19716 ts_tablet_manager.cc:1929] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251023 20:16:47.545717 19716 log.cc:1199] T f4ff9ca441b14a468709c0efd0ae984e P cea0984de40845da94e6f23513009f02: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/f4ff9ca441b14a468709c0efd0ae984e
I20251023 20:16:47.546371 18938 catalog_manager.cc:4985] TS cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): tablet f4ff9ca441b14a468709c0efd0ae984e (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:47.557058 19689 raft_consensus.cc:1064] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 329b90f727b745ddacc774c89c3a4428 to VOTER
I20251023 20:16:47.557641 19689 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:47.559846 19054 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.560004 19598 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.559882 19128 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:16:47.560474 19685 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.560690 19672 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.560942 19687 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:47.565840 19687 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.566532 19128 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.566851 19054 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.570328 19598 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.573374 18951 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 28 to 29, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:47.577631 19202 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:47.579037 19598 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.579155 19054 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.579631 19689 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.579864 19685 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.581928 19687 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.582569 19598 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.582728 19054 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:47.585501 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 887ca8402e81417c94da83fdc1b75862 with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:47.586153 18951 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 29 to 30, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:47.589633 19113 tablet_service.cc:1552] Processing DeleteTablet for tablet 887ca8402e81417c94da83fdc1b75862 with delete_type TABLET_DATA_TOMBSTONED (TS f909677d13e94318ad9a3b1b2db05723 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:36224
I20251023 20:16:47.589898 19713 tablet_replica.cc:333] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:16:47.590099 19713 raft_consensus.cc:2243] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:47.590312 19713 raft_consensus.cc:2272] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:47.591156 19713 ts_tablet_manager.cc:1916] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:47.594805 19713 ts_tablet_manager.cc:1929] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.29
I20251023 20:16:47.594882 19713 log.cc:1199] T 887ca8402e81417c94da83fdc1b75862 P f909677d13e94318ad9a3b1b2db05723: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/887ca8402e81417c94da83fdc1b75862
I20251023 20:16:47.595580 18939 catalog_manager.cc:4985] TS f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): tablet 887ca8402e81417c94da83fdc1b75862 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:47.704437 19721 raft_consensus.cc:1064] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: attempting to promote NON_VOTER 3c36454265a141c0adf91706fa3115e9 to VOTER
I20251023 20:16:47.705199 19721 consensus_queue.cc:237] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:47.707527 19202 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.707541 19525 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.707624 19054 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:16:47.708065 19686 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.708943 19686 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.709204 19722 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:47.716648 19721 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.717319 19202 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.718338 19525 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.721544 19054 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.721901 18951 catalog_manager.cc:5649] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 29 to 30, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.726310 19128 consensus_queue.cc:237] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:47.728592 19525 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:47.729336 19724 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:47.729779 19054 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:47.730805 19722 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:47.731621 19724 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.732161 19525 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.733821 19054 raft_consensus.cc:2955] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:47.734632 18939 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 508756851fc14f93a7d310b11beb15cb with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:47.735651 18951 catalog_manager.cc:5649] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New cstate: current_term: 1 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:47.739899 19196 tablet_service.cc:1552] Processing DeleteTablet for tablet 508756851fc14f93a7d310b11beb15cb with delete_type TABLET_DATA_TOMBSTONED (TS cea0984de40845da94e6f23513009f02 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:47.740188 19716 tablet_replica.cc:333] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:16:47.740377 19716 raft_consensus.cc:2243] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:47.740641 19716 raft_consensus.cc:2272] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:47.741426 19716 ts_tablet_manager.cc:1916] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:47.745273 19716 ts_tablet_manager.cc:1929] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251023 20:16:47.745357 19716 log.cc:1199] T 508756851fc14f93a7d310b11beb15cb P cea0984de40845da94e6f23513009f02: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/508756851fc14f93a7d310b11beb15cb
I20251023 20:16:47.745879 18938 catalog_manager.cc:4985] TS cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): tablet 508756851fc14f93a7d310b11beb15cb (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:48.105959 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:48.108669 19202 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
new_leader_uuid: "3c36454265a141c0adf91706fa3115e9"
from {username='slave'} at 127.0.0.1:53226
I20251023 20:16:48.108832 19202 raft_consensus.cc:606] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Received request to transfer leadership to TS 3c36454265a141c0adf91706fa3115e9
I20251023 20:16:48.117219 19054 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "37a00c71a3934405917c05725c94dde5"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "329b90f727b745ddacc774c89c3a4428"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:48.117359 19054 raft_consensus.cc:606] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Received request to transfer leadership to TS 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:48.121981 19131 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:16:48.122139 19131 raft_consensus.cc:606] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:48.122962 19131 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:16:48.123090 19131 raft_consensus.cc:606] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Received request to transfer leadership to TS cea0984de40845da94e6f23513009f02
I20251023 20:16:48.123960 19054 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:48.124071 19054 raft_consensus.cc:606] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Received request to transfer leadership to TS 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:48.124831 19202 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:53226
I20251023 20:16:48.124974 19202 raft_consensus.cc:606] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:48.125317 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20251023 20:16:48.125530 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:48.141969 19732 raft_consensus.cc:993] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: : Instructing follower 3c36454265a141c0adf91706fa3115e9 to start an election
I20251023 20:16:48.142102 19732 raft_consensus.cc:1081] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Signalling peer 3c36454265a141c0adf91706fa3115e9 to start an election
I20251023 20:16:48.142666 19525 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e"
dest_uuid: "3c36454265a141c0adf91706fa3115e9"
from {username='slave'} at 127.0.0.1:33592
I20251023 20:16:48.142787 19525 raft_consensus.cc:493] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:48.142882 19525 raft_consensus.cc:3060] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.144662 19525 raft_consensus.cc:515] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:48.145315 19525 leader_election.cc:290] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [CANDIDATE]: Term 2 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:48.145706 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "3c36454265a141c0adf91706fa3115e9" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:48.145769 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a9aada4eefa459ea9801d5506a2b82e" candidate_uuid: "3c36454265a141c0adf91706fa3115e9" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:48.145876 19131 raft_consensus.cc:3060] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.145875 19202 raft_consensus.cc:3055] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:48.146098 19202 raft_consensus.cc:740] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:48.146358 19202 consensus_queue.cc:260] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:48.146618 19202 raft_consensus.cc:3060] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.148411 19131 raft_consensus.cc:2468] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c36454265a141c0adf91706fa3115e9 in term 2.
I20251023 20:16:48.148604 19128 raft_consensus.cc:1240] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Rejecting Update request from peer cea0984de40845da94e6f23513009f02 for earlier term 1. Current term is 2. Ops: []
I20251023 20:16:48.148821 19488 leader_election.cc:304] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c36454265a141c0adf91706fa3115e9, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:48.149012 19202 raft_consensus.cc:2468] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c36454265a141c0adf91706fa3115e9 in term 2.
I20251023 20:16:48.149111 19728 raft_consensus.cc:2804] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:48.149503 19728 raft_consensus.cc:697] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 LEADER]: Becoming Leader. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Running, Role: LEADER
I20251023 20:16:48.149734 19728 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:48.151482 18952 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 reported cstate change: term changed from 1 to 2, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to 3c36454265a141c0adf91706fa3115e9 (127.15.81.197). New cstate: current_term: 2 leader_uuid: "3c36454265a141c0adf91706fa3115e9" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:48.195624 19722 raft_consensus.cc:993] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:48.195801 19722 raft_consensus.cc:1081] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:48.196417 19054 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:16:48.196635 19054 raft_consensus.cc:493] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:48.196718 19054 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.198559 19687 raft_consensus.cc:993] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:48.198704 19732 raft_consensus.cc:1081] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:48.199065 19054 raft_consensus.cc:515] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:48.199318 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55936
I20251023 20:16:48.199481 19056 raft_consensus.cc:493] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:48.199565 19056 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.199702 19054 leader_election.cc:290] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 2 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571)
I20251023 20:16:48.201164 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:48.201318 19128 raft_consensus.cc:3055] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:48.201377 19128 raft_consensus.cc:740] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:48.201603 19128 consensus_queue.cc:260] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:48.201804 19128 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.202894 19056 raft_consensus.cc:515] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:48.203510 19128 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 2.
I20251023 20:16:48.203997 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:48.204155 19128 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.204286 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:48.204478 19202 raft_consensus.cc:3055] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:48.204547 19202 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:48.205008 19020 leader_election.cc:304] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:48.204797 19202 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:48.205250 19702 raft_consensus.cc:2804] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:48.205340 19202 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.205626 19702 raft_consensus.cc:697] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:48.205861 19702 consensus_queue.cc:237] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:48.206740 19128 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 2.
I20251023 20:16:48.207072 19020 leader_election.cc:304] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:48.207341 19702 raft_consensus.cc:2804] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:48.207446 19702 raft_consensus.cc:697] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:48.207619 19702 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:48.208252 19202 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 2.
I20251023 20:16:48.209524 19056 leader_election.cc:290] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 2 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:48.211948 18952 catalog_manager.cc:5649] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 1 to 2, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 2 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:48.212379 19525 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "3c36454265a141c0adf91706fa3115e9"
I20251023 20:16:48.212373 18952 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 1 to 2, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 2 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:48.212589 19525 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.214139 19525 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 2.
I20251023 20:16:48.249223 19724 raft_consensus.cc:993] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: : Instructing follower cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:48.249378 19724 raft_consensus.cc:1081] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Signalling peer cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:48.249891 19202 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:53212
I20251023 20:16:48.250048 19202 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:48.250137 19202 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.251892 19202 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:48.252410 19202 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 2 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:48.252720 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:48.252820 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:48.252928 19056 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.253023 19128 raft_consensus.cc:3055] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:48.253095 19128 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:48.253324 19128 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:48.253597 19128 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.254635 19056 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 2.
I20251023 20:16:48.254987 19172 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:48.255203 19732 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:48.255303 19732 raft_consensus.cc:697] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:48.255338 19128 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 2.
I20251023 20:16:48.255537 19732 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:48.257881 18952 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 1 to 2, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 2 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
I20251023 20:16:48.512904 19736 raft_consensus.cc:993] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower 329b90f727b745ddacc774c89c3a4428 to start an election
I20251023 20:16:48.513047 19736 raft_consensus.cc:1081] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Signalling peer 329b90f727b745ddacc774c89c3a4428 to start an election
I20251023 20:16:48.513615 19598 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "37a00c71a3934405917c05725c94dde5"
dest_uuid: "329b90f727b745ddacc774c89c3a4428"
from {username='slave'} at 127.0.0.1:49354
I20251023 20:16:48.513773 19598 raft_consensus.cc:493] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:48.513856 19598 raft_consensus.cc:3060] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.515545 19598 raft_consensus.cc:515] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:48.516184 19598 leader_election.cc:290] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [CANDIDATE]: Term 2 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:48.516561 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37a00c71a3934405917c05725c94dde5" candidate_uuid: "329b90f727b745ddacc774c89c3a4428" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:48.516618 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37a00c71a3934405917c05725c94dde5" candidate_uuid: "329b90f727b745ddacc774c89c3a4428" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:48.516706 19056 raft_consensus.cc:3055] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:48.516770 19056 raft_consensus.cc:740] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:48.516796 19202 raft_consensus.cc:3060] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.517014 19056 consensus_queue.cc:260] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:48.517273 19056 raft_consensus.cc:3060] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.518512 19202 raft_consensus.cc:2468] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329b90f727b745ddacc774c89c3a4428 in term 2.
I20251023 20:16:48.518819 19562 leader_election.cc:304] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329b90f727b745ddacc774c89c3a4428, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:48.519006 19056 raft_consensus.cc:2468] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329b90f727b745ddacc774c89c3a4428 in term 2.
I20251023 20:16:48.519294 19739 raft_consensus.cc:2804] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:48.519619 19739 raft_consensus.cc:697] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEADER]: Becoming Leader. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Running, Role: LEADER
I20251023 20:16:48.519840 19739 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:48.522181 18952 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: term changed from 1 to 2, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to 329b90f727b745ddacc774c89c3a4428 (127.15.81.198). New cstate: current_term: 2 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:48.601330 19202 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 20:16:48.601827 19728 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:48.603745 19702 raft_consensus.cc:993] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower 4cccadfc20fb475c9e6c1ce6cd3ba850 to start an election
I20251023 20:16:48.603891 19736 raft_consensus.cc:1081] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Signalling peer 4cccadfc20fb475c9e6c1ce6cd3ba850 to start an election
I20251023 20:16:48.604455 19449 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e"
dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
from {username='slave'} at 127.0.0.1:48954
I20251023 20:16:48.604597 19449 raft_consensus.cc:493] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:48.604671 19449 raft_consensus.cc:3060] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.607080 19128 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 20:16:48.607430 19449 raft_consensus.cc:515] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:48.609114 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" candidate_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:48.609308 19056 raft_consensus.cc:3055] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:48.609382 19056 raft_consensus.cc:740] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:48.609658 19056 consensus_queue.cc:260] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:48.610045 19056 raft_consensus.cc:3060] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.610477 19449 leader_election.cc:290] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [CANDIDATE]: Term 2 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:48.610692 19728 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:48.613780 19056 raft_consensus.cc:2468] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cccadfc20fb475c9e6c1ce6cd3ba850 in term 2.
I20251023 20:16:48.614491 19415 leader_election.cc:304] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, 4cccadfc20fb475c9e6c1ce6cd3ba850; no voters:
I20251023 20:16:48.618949 19748 raft_consensus.cc:2804] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:48.623569 19131 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251023 20:16:48.624162 19736 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:48.627432 19748 raft_consensus.cc:697] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Becoming Leader. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Running, Role: LEADER
I20251023 20:16:48.627811 19748 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:48.631767 19202 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251023 20:16:48.632225 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" candidate_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:48.632477 19128 raft_consensus.cc:3060] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:48.632721 19736 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:48.635541 19128 raft_consensus.cc:2468] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cccadfc20fb475c9e6c1ce6cd3ba850 in term 2.
I20251023 20:16:48.637509 18951 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: term changed from 1 to 2, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196). New cstate: current_term: 2 leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:48.650832 19128 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:48.651248 19702 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:48.653995 19128 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:48.654476 19732 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:48.654666 19525 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:48.656649 19702 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:48.659858 19056 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:48.661202 19758 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:48.740574 19202 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:48.742900 19128 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:48.742950 19449 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
W20251023 20:16:48.743108 19172 consensus_peers.cc:597] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 -> Peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571): Couldn't send request to peer 3c36454265a141c0adf91706fa3115e9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 461a7ab7c21c40fbb4ef24809dad5831. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:48.743525 19732 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:48.743748 19758 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:48.746089 19687 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:48.746737 19449 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:48.750161 19128 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:48.750623 18946 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 30 to 31, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:48.751545 19202 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:48.756108 19598 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:48.756158 19056 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251023 20:16:48.756543 19687 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
W20251023 20:16:48.756764 19170 consensus_peers.cc:597] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 -> Peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839): Couldn't send request to peer 4cccadfc20fb475c9e6c1ce6cd3ba850. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 887ca8402e81417c94da83fdc1b75862. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:48.756764 19732 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:48.759095 19732 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:48.759626 19056 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:48.760180 19598 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:48.763583 19054 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:48.764365 18951 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index 30 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:48.766072 19202 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:48.766081 19128 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
W20251023 20:16:48.766489 19020 consensus_peers.cc:597] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38153cd2d04e4c56b22f87e07a3c2d11. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:48.766659 19755 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:48.766872 19756 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:48.769285 19755 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:48.773641 19202 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:48.774987 18951 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:48.784420 19128 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:48.797274 19525 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:48.798997 19202 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:48.799571 19745 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:48.799703 19128 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:48.800071 19745 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:48.802073 19728 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 30 to 32, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:48.802686 19202 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:48.804229 19128 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 30 to 32, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
W20251023 20:16:48.807132 19488 consensus_peers.cc:597] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 -> Peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839): Couldn't send request to peer 4cccadfc20fb475c9e6c1ce6cd3ba850. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a9aada4eefa459ea9801d5506a2b82e. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:48.808295 18951 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 reported cstate change: config changed from index 30 to 32, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New cstate: current_term: 2 leader_uuid: "3c36454265a141c0adf91706fa3115e9" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:48.817435 19002 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251023 20:16:48.823014 19002 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 37a00c71a3934405917c05725c94dde5, TS 116b80452edd43bc9d822d31be0c5c38 -> TS 3c36454265a141c0adf91706fa3115e9 move failed, destination replica disappeared from tablet's Raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
W20251023 20:16:48.823074 19002 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 37a00c71a3934405917c05725c94dde5, TS 116b80452edd43bc9d822d31be0c5c38 -> TS 3c36454265a141c0adf91706fa3115e9 move failed, destination replica disappeared from tablet's Raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
W20251023 20:16:48.828274 19002 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet b4c390009f384f9f9a3f424a69b2e5ce, TS cea0984de40845da94e6f23513009f02 -> TS 329b90f727b745ddacc774c89c3a4428 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
W20251023 20:16:48.828336 19002 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet b4c390009f384f9f9a3f424a69b2e5ce, TS cea0984de40845da94e6f23513009f02 -> TS 329b90f727b745ddacc774c89c3a4428 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:49.019464 19202 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:49.019953 19739 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:49.025032 19056 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:49.025734 19775 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:49.090381 19128 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:49.090893 19748 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:49.094832 19054 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251023 20:16:49.095335 19748 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:49.221508 19781 ts_tablet_manager.cc:933] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: Initiating tablet copy from peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:49.222275 19781 tablet_copy_client.cc:323] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:37799
I20251023 20:16:49.222805 19216 tablet_copy_service.cc:140] P cea0984de40845da94e6f23513009f02: Received BeginTabletCopySession request for tablet 461a7ab7c21c40fbb4ef24809dad5831 from peer 3c36454265a141c0adf91706fa3115e9 ({username='slave'} at 127.0.0.1:53250)
I20251023 20:16:49.222920 19216 tablet_copy_service.cc:161] P cea0984de40845da94e6f23513009f02: Beginning new tablet copy session on tablet 461a7ab7c21c40fbb4ef24809dad5831 from peer 3c36454265a141c0adf91706fa3115e9 at {username='slave'} at 127.0.0.1:53250: session id = 3c36454265a141c0adf91706fa3115e9-461a7ab7c21c40fbb4ef24809dad5831
I20251023 20:16:49.225589 19216 tablet_copy_source_session.cc:215] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:49.226619 19781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 461a7ab7c21c40fbb4ef24809dad5831. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:49.231910 19781 tablet_copy_client.cc:806] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:49.232225 19781 tablet_copy_client.cc:670] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:49.233590 19782 tablet_copy_client.cc:1025] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:49.293479 19783 ts_tablet_manager.cc:933] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Initiating tablet copy from peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:49.294170 19783 tablet_copy_client.cc:323] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:37799
I20251023 20:16:49.294737 19216 tablet_copy_service.cc:140] P cea0984de40845da94e6f23513009f02: Received BeginTabletCopySession request for tablet 887ca8402e81417c94da83fdc1b75862 from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 ({username='slave'} at 127.0.0.1:58466)
I20251023 20:16:49.294867 19216 tablet_copy_service.cc:161] P cea0984de40845da94e6f23513009f02: Beginning new tablet copy session on tablet 887ca8402e81417c94da83fdc1b75862 from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 at {username='slave'} at 127.0.0.1:58466: session id = 4cccadfc20fb475c9e6c1ce6cd3ba850-887ca8402e81417c94da83fdc1b75862
I20251023 20:16:49.297416 19216 tablet_copy_source_session.cc:215] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:49.298592 19783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887ca8402e81417c94da83fdc1b75862. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:49.299124 19785 ts_tablet_manager.cc:933] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Initiating tablet copy from peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571)
I20251023 20:16:49.299988 19785 tablet_copy_client.cc:323] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.197:33571
I20251023 20:16:49.305003 19783 tablet_copy_client.cc:806] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:49.305266 19783 tablet_copy_client.cc:670] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:49.307251 19784 tablet_copy_client.cc:1025] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:49.311825 19535 tablet_copy_service.cc:140] P 3c36454265a141c0adf91706fa3115e9: Received BeginTabletCopySession request for tablet 2a9aada4eefa459ea9801d5506a2b82e from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 ({username='slave'} at 127.0.0.1:38436)
I20251023 20:16:49.311947 19535 tablet_copy_service.cc:161] P 3c36454265a141c0adf91706fa3115e9: Beginning new tablet copy session on tablet 2a9aada4eefa459ea9801d5506a2b82e from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 at {username='slave'} at 127.0.0.1:38436: session id = 4cccadfc20fb475c9e6c1ce6cd3ba850-2a9aada4eefa459ea9801d5506a2b82e
I20251023 20:16:49.314044 19535 tablet_copy_source_session.cc:215] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:49.315013 19785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9aada4eefa459ea9801d5506a2b82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:49.319986 19785 tablet_copy_client.cc:806] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:49.320218 19785 tablet_copy_client.cc:670] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:49.321573 19786 tablet_copy_client.cc:1025] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:49.330335 19789 ts_tablet_manager.cc:933] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Initiating tablet copy from peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:49.330850 19789 tablet_copy_client.cc:323] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:36987
I20251023 20:16:49.331398 19067 tablet_copy_service.cc:140] P 116b80452edd43bc9d822d31be0c5c38: Received BeginTabletCopySession request for tablet 38153cd2d04e4c56b22f87e07a3c2d11 from peer 329b90f727b745ddacc774c89c3a4428 ({username='slave'} at 127.0.0.1:49630)
I20251023 20:16:49.331521 19067 tablet_copy_service.cc:161] P 116b80452edd43bc9d822d31be0c5c38: Beginning new tablet copy session on tablet 38153cd2d04e4c56b22f87e07a3c2d11 from peer 329b90f727b745ddacc774c89c3a4428 at {username='slave'} at 127.0.0.1:49630: session id = 329b90f727b745ddacc774c89c3a4428-38153cd2d04e4c56b22f87e07a3c2d11
I20251023 20:16:49.333905 19067 tablet_copy_source_session.cc:215] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:49.334807 19789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38153cd2d04e4c56b22f87e07a3c2d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:49.339844 19789 tablet_copy_client.cc:806] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:49.340060 19789 tablet_copy_client.cc:670] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:49.341398 19790 tablet_copy_client.cc:1025] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:50.125861 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:50.129109 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:50.129323 19056 raft_consensus.cc:606] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:50.130256 19203 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:53226
I20251023 20:16:50.130401 19203 raft_consensus.cc:606] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:50.131296 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:50.131489 19056 raft_consensus.cc:606] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:50.131851 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 20:16:50.132054 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:50.188344 19795 raft_consensus.cc:993] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:50.188548 19795 raft_consensus.cc:1081] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:50.189122 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:16:50.189337 19128 raft_consensus.cc:493] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:50.189465 19128 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:50.191699 19128 raft_consensus.cc:515] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:50.192451 19128 leader_election.cc:290] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 3 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:50.192623 19203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:50.192802 19203 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:50.193146 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:50.193356 19056 raft_consensus.cc:3055] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:16:50.193456 19056 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:50.193714 19056 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:50.194080 19056 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:50.195081 19203 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 3.
I20251023 20:16:50.195501 19095 leader_election.cc:304] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:50.195837 19724 raft_consensus.cc:2804] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:16:50.196260 19724 raft_consensus.cc:697] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:50.196559 19724 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:50.196882 19056 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 3.
I20251023 20:16:50.199222 18952 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 2 to 3, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 3 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:50.411970 19795 raft_consensus.cc:993] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:50.412120 19795 raft_consensus.cc:1081] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:50.412634 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:16:50.412802 19128 raft_consensus.cc:493] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:50.412916 19128 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:50.415190 19128 raft_consensus.cc:515] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:50.415867 19128 leader_election.cc:290] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 2 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:50.416226 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:50.416455 19202 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:50.416416 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:50.416616 19056 raft_consensus.cc:3055] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:50.416693 19056 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:50.416952 19056 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:50.417315 19056 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:50.418469 19202 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 2.
I20251023 20:16:50.418849 19095 leader_election.cc:304] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:50.419109 19724 raft_consensus.cc:2804] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:50.419219 19724 raft_consensus.cc:697] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:50.419400 19056 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 2.
I20251023 20:16:50.419409 19724 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:50.422013 18952 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 1 to 2, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:50.507535 19793 raft_consensus.cc:993] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:50.507706 19793 raft_consensus.cc:1081] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:50.508221 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55936
I20251023 20:16:50.508385 19056 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:50.508495 19056 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:50.510247 19056 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:50.510699 19056 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 3 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:50.511036 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:50.511039 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:50.511221 19128 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:50.511219 19202 raft_consensus.cc:3055] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:16:50.511308 19202 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:50.511519 19202 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:50.511778 19202 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:50.513643 19128 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 3.
I20251023 20:16:50.513999 19020 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:50.514130 19202 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 3.
I20251023 20:16:50.514249 19795 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:16:50.514370 19795 raft_consensus.cc:697] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:50.514586 19795 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:50.516935 18952 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 2 to 3, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 3 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:50.758651 19056 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251023 20:16:50.759131 19724 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:50.764662 19202 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251023 20:16:50.765339 19724 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:50.819670 19056 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:50.820199 19800 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:50.824194 19203 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:50.824725 19800 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251023 20:16:50.848455 19096 consensus_peers.cc:597] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:50.947532 19202 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 20:16:50.948004 19795 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:50.951841 19131 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 20:16:50.952374 19795 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:51.233956 19781 tablet_copy_client.cc:538] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:51.236773 19781 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: Bootstrap starting.
I20251023 20:16:51.274189 19781 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=95 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:51.274545 19781 tablet_bootstrap.cc:492] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: Bootstrap complete.
I20251023 20:16:51.274785 19781 ts_tablet_manager.cc:1403] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: Time spent bootstrapping tablet: real 0.038s user 0.030s sys 0.008s
I20251023 20:16:51.275401 19781 raft_consensus.cc:359] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:51.275547 19781 raft_consensus.cc:740] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Initialized, Role: LEARNER
I20251023 20:16:51.275741 19781 consensus_queue.cc:260] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:51.276244 19781 ts_tablet_manager.cc:1434] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:51.276991 19216 tablet_copy_service.cc:342] P cea0984de40845da94e6f23513009f02: Request end of tablet copy session 3c36454265a141c0adf91706fa3115e9-461a7ab7c21c40fbb4ef24809dad5831 received from {username='slave'} at 127.0.0.1:53250
I20251023 20:16:51.277100 19216 tablet_copy_service.cc:434] P cea0984de40845da94e6f23513009f02: ending tablet copy session 3c36454265a141c0adf91706fa3115e9-461a7ab7c21c40fbb4ef24809dad5831 on tablet 461a7ab7c21c40fbb4ef24809dad5831 with peer 3c36454265a141c0adf91706fa3115e9
I20251023 20:16:51.307559 19783 tablet_copy_client.cc:538] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:51.310668 19783 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap starting.
I20251023 20:16:51.314939 19525 raft_consensus.cc:1217] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.30->[1.31-1.31] Dedup: 1.31->[]
I20251023 20:16:51.321893 19785 tablet_copy_client.cc:538] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:51.325060 19785 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap starting.
I20251023 20:16:51.341707 19789 tablet_copy_client.cc:538] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:51.343927 19789 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Bootstrap starting.
I20251023 20:16:51.352274 19783 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=127 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:51.352666 19783 tablet_bootstrap.cc:492] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap complete.
I20251023 20:16:51.352933 19783 ts_tablet_manager.cc:1403] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent bootstrapping tablet: real 0.042s user 0.041s sys 0.002s
I20251023 20:16:51.353650 19783 raft_consensus.cc:359] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:51.353911 19783 raft_consensus.cc:740] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Initialized, Role: LEARNER
I20251023 20:16:51.354140 19783 consensus_queue.cc:260] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:51.355182 19783 ts_tablet_manager.cc:1434] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:16:51.355775 19216 tablet_copy_service.cc:342] P cea0984de40845da94e6f23513009f02: Request end of tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-887ca8402e81417c94da83fdc1b75862 received from {username='slave'} at 127.0.0.1:58466
I20251023 20:16:51.355878 19216 tablet_copy_service.cc:434] P cea0984de40845da94e6f23513009f02: ending tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-887ca8402e81417c94da83fdc1b75862 on tablet 887ca8402e81417c94da83fdc1b75862 with peer 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:51.372365 19785 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=110 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:51.372978 19785 tablet_bootstrap.cc:492] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap complete.
I20251023 20:16:51.373307 19785 ts_tablet_manager.cc:1403] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent bootstrapping tablet: real 0.048s user 0.035s sys 0.012s
I20251023 20:16:51.373900 19785 raft_consensus.cc:359] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:51.374055 19785 raft_consensus.cc:740] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Initialized, Role: LEARNER
I20251023 20:16:51.374225 19785 consensus_queue.cc:260] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:51.374724 19785 ts_tablet_manager.cc:1434] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:16:51.375260 19535 tablet_copy_service.cc:342] P 3c36454265a141c0adf91706fa3115e9: Request end of tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-2a9aada4eefa459ea9801d5506a2b82e received from {username='slave'} at 127.0.0.1:38436
I20251023 20:16:51.375365 19535 tablet_copy_service.cc:434] P 3c36454265a141c0adf91706fa3115e9: ending tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-2a9aada4eefa459ea9801d5506a2b82e on tablet 2a9aada4eefa459ea9801d5506a2b82e with peer 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:51.380232 19789 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:51.380548 19789 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Bootstrap complete.
I20251023 20:16:51.380736 19789 ts_tablet_manager.cc:1403] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Time spent bootstrapping tablet: real 0.037s user 0.021s sys 0.012s
I20251023 20:16:51.381402 19789 raft_consensus.cc:359] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:51.381587 19789 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Initialized, Role: LEARNER
I20251023 20:16:51.381771 19789 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:51.382220 19789 ts_tablet_manager.cc:1434] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20251023 20:16:51.382715 19067 tablet_copy_service.cc:342] P 116b80452edd43bc9d822d31be0c5c38: Request end of tablet copy session 329b90f727b745ddacc774c89c3a4428-38153cd2d04e4c56b22f87e07a3c2d11 received from {username='slave'} at 127.0.0.1:49630
I20251023 20:16:51.382797 19067 tablet_copy_service.cc:434] P 116b80452edd43bc9d822d31be0c5c38: ending tablet copy session 329b90f727b745ddacc774c89c3a4428-38153cd2d04e4c56b22f87e07a3c2d11 on tablet 38153cd2d04e4c56b22f87e07a3c2d11 with peer 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:51.703049 19598 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 1 LEARNER]: Advancing to term 2
I20251023 20:16:51.831110 19830 raft_consensus.cc:1064] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 3c36454265a141c0adf91706fa3115e9 to VOTER
I20251023 20:16:51.832145 19830 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:51.834575 19525 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 20:16:51.834633 19449 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 20:16:51.835067 19793 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:51.835268 19832 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:51.835414 19131 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 20:16:51.835911 19830 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:51.838390 19793 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.841753 19525 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.842209 19449 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.846432 18951 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 31 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.848651 19449 raft_consensus.cc:1217] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.31->[2.32-2.32] Dedup: 2.32->[]
I20251023 20:16:51.850988 19131 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.866547 19203 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:51.868368 19449 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 20:16:51.868376 19525 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 20:16:51.868830 19830 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:51.869206 19793 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:51.871189 19832 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.872063 19449 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.874162 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 461a7ab7c21c40fbb4ef24809dad5831 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:51.874536 19525 raft_consensus.cc:2955] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:51.875491 18946 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:51.879796 19113 tablet_service.cc:1552] Processing DeleteTablet for tablet 461a7ab7c21c40fbb4ef24809dad5831 with delete_type TABLET_DATA_TOMBSTONED (TS f909677d13e94318ad9a3b1b2db05723 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:36224
I20251023 20:16:51.889138 19840 tablet_replica.cc:333] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:16:51.889302 19840 raft_consensus.cc:2243] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:51.889643 19840 raft_consensus.cc:2272] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:51.890338 19840 ts_tablet_manager.cc:1916] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:51.896456 19840 ts_tablet_manager.cc:1929] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.32
I20251023 20:16:51.896566 19840 log.cc:1199] T 461a7ab7c21c40fbb4ef24809dad5831 P f909677d13e94318ad9a3b1b2db05723: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/461a7ab7c21c40fbb4ef24809dad5831
I20251023 20:16:51.897359 18939 catalog_manager.cc:4985] TS f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): tablet 461a7ab7c21c40fbb4ef24809dad5831 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:51.925798 19449 raft_consensus.cc:1217] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.30->[1.31-1.31] Dedup: 1.31->[]
I20251023 20:16:52.132413 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:52.135324 19131 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:16:52.135512 19131 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:52.136453 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:52.136648 19056 raft_consensus.cc:606] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Received request to transfer leadership to TS cea0984de40845da94e6f23513009f02
I20251023 20:16:52.137645 19131 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:16:52.147292 19131 raft_consensus.cc:606] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:52.147717 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 20:16:52.147938 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:52.227020 19841 raft_consensus.cc:993] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:52.227236 19841 raft_consensus.cc:1081] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Signalling peer cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:52.227823 19202 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:53202
I20251023 20:16:52.228005 19202 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:52.228096 19202 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:52.230993 19202 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.232080 19202 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 4 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:52.232407 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:52.232579 19056 raft_consensus.cc:3055] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 20:16:52.232654 19056 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:52.232897 19056 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.233151 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:52.233212 19056 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:52.233296 19131 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:52.235652 19056 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 4.
I20251023 20:16:52.235826 19131 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 4.
I20251023 20:16:52.236080 19172 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:52.236331 19793 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 20:16:52.236454 19793 raft_consensus.cc:697] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:52.236653 19793 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.239027 18946 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 3 to 4, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } }
I20251023 20:16:52.266072 19843 raft_consensus.cc:1064] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: attempting to promote NON_VOTER 329b90f727b745ddacc774c89c3a4428 to VOTER
I20251023 20:16:52.266749 19843 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 28, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:52.269222 19598 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 20:16:52.269376 19056 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 20:16:52.269862 19800 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:52.269974 19203 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 20:16:52.270313 19800 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:52.270800 19843 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:52.274585 19202 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 28 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.274552 19056 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 28 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.274060 19844 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index 28 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.278685 18946 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 28 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.280654 19598 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 28 to 30, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.286049 19131 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:52.287467 19056 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 20:16:52.287564 19598 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 20:16:52.288128 19844 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:52.288538 19844 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:52.290099 19848 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.290601 19598 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.293370 18939 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 38153cd2d04e4c56b22f87e07a3c2d11 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:52.294453 18946 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New cstate: current_term: 2 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.296660 19056 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:52.297827 19196 tablet_service.cc:1552] Processing DeleteTablet for tablet 38153cd2d04e4c56b22f87e07a3c2d11 with delete_type TABLET_DATA_TOMBSTONED (TS cea0984de40845da94e6f23513009f02 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:52.310045 19849 tablet_replica.cc:333] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:16:52.310236 19849 raft_consensus.cc:2243] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:52.310534 19849 raft_consensus.cc:2272] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:52.311254 19849 ts_tablet_manager.cc:1916] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:52.315634 19849 ts_tablet_manager.cc:1929] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.30
I20251023 20:16:52.315748 19849 log.cc:1199] T 38153cd2d04e4c56b22f87e07a3c2d11 P cea0984de40845da94e6f23513009f02: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/38153cd2d04e4c56b22f87e07a3c2d11
I20251023 20:16:52.316349 18938 catalog_manager.cc:4985] TS cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): tablet 38153cd2d04e4c56b22f87e07a3c2d11 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:52.368772 19851 raft_consensus.cc:1064] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: attempting to promote NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 to VOTER
I20251023 20:16:52.369482 19851 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 30, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:52.372493 19449 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEARNER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:52.373081 19202 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:52.372987 19850 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:52.373525 19128 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:52.373998 19793 raft_consensus.cc:1064] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 to VOTER
I20251023 20:16:52.374349 19850 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:52.374707 19793 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:52.375200 19853 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:52.377753 19853 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.378937 19449 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.381531 19202 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.381726 19128 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.382279 18952 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 reported cstate change: config changed from index 32 to 33, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3c36454265a141c0adf91706fa3115e9" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:52.388159 19525 consensus_queue.cc:237] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:52.388654 19449 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 20:16:52.389747 19448 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:52.390319 19830 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:52.390661 19850 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:52.390972 19203 raft_consensus.cc:1275] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 3c36454265a141c0adf91706fa3115e9: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:52.392139 19850 consensus_queue.cc:1048] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:52.393854 19850 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.396404 19001 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 2a9aada4eefa459ea9801d5506a2b82e with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:52.398334 18952 catalog_manager.cc:5649] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 reported cstate change: config changed from index 33 to 34, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New cstate: current_term: 2 leader_uuid: "3c36454265a141c0adf91706fa3115e9" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:52.399066 19598 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 20:16:52.399552 19793 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.001s
I20251023 20:16:52.400617 19056 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251023 20:16:52.401731 19858 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:52.401266 19202 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.402613 19113 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a9aada4eefa459ea9801d5506a2b82e with delete_type TABLET_DATA_TOMBSTONED (TS f909677d13e94318ad9a3b1b2db05723 not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:36224
I20251023 20:16:52.408465 19858 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.408661 19448 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.409338 19449 raft_consensus.cc:2955] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.411660 19056 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.413163 18952 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:52.414268 19598 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.418740 19203 consensus_queue.cc:237] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:52.421693 19598 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 20:16:52.422151 19793 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:52.425105 19861 tablet_replica.cc:333] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:16:52.425243 19448 raft_consensus.cc:1275] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251023 20:16:52.425365 19861 raft_consensus.cc:2243] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:52.425843 19861 raft_consensus.cc:2272] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:52.425956 19832 consensus_queue.cc:1048] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:52.426156 19830 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.426801 19861 ts_tablet_manager.cc:1916] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:52.429580 19448 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.430331 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 887ca8402e81417c94da83fdc1b75862 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:52.430590 18952 catalog_manager.cc:5649] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:52.431239 19598 raft_consensus.cc:2955] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:52.432269 19861 ts_tablet_manager.cc:1929] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.33
I20251023 20:16:52.432395 19861 log.cc:1199] T 2a9aada4eefa459ea9801d5506a2b82e P f909677d13e94318ad9a3b1b2db05723: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/2a9aada4eefa459ea9801d5506a2b82e
I20251023 20:16:52.433005 18939 catalog_manager.cc:4985] TS f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): tablet 2a9aada4eefa459ea9801d5506a2b82e (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:52.433944 19041 tablet_service.cc:1552] Processing DeleteTablet for tablet 887ca8402e81417c94da83fdc1b75862 with delete_type TABLET_DATA_TOMBSTONED (TS 116b80452edd43bc9d822d31be0c5c38 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:52.434553 19863 tablet_replica.cc:333] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:16:52.434727 19863 raft_consensus.cc:2243] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:52.435565 19863 raft_consensus.cc:2272] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:52.436355 19863 ts_tablet_manager.cc:1916] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:52.440127 19863 ts_tablet_manager.cc:1929] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.32
I20251023 20:16:52.440214 19863 log.cc:1199] T 887ca8402e81417c94da83fdc1b75862 P 116b80452edd43bc9d822d31be0c5c38: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/887ca8402e81417c94da83fdc1b75862
I20251023 20:16:52.440717 18941 catalog_manager.cc:4985] TS 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): tablet 887ca8402e81417c94da83fdc1b75862 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:52.456275 19844 raft_consensus.cc:993] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:52.456413 19844 raft_consensus.cc:1081] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:52.456988 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:16:52.457123 19056 raft_consensus.cc:493] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:52.457209 19056 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:52.458771 19056 raft_consensus.cc:515] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.459200 19056 leader_election.cc:290] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 4 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:52.459561 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:52.459618 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc60b796f0d946fbb49d15812e88ac60" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:52.459693 19128 raft_consensus.cc:3055] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 20:16:52.459751 19202 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:52.459760 19128 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:52.459985 19128 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.460211 19128 raft_consensus.cc:3060] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:52.461676 19202 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 4.
I20251023 20:16:52.462040 19128 raft_consensus.cc:2468] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 4.
I20251023 20:16:52.462038 19019 leader_election.cc:304] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:52.462284 19841 raft_consensus.cc:2804] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 20:16:52.462432 19841 raft_consensus.cc:697] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:52.462617 19841 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.464648 18952 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 3 to 4, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 4 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:52.699458 19843 raft_consensus.cc:993] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:52.699615 19843 raft_consensus.cc:1081] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:52.700084 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:16:52.700237 19056 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:52.700320 19056 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:52.702135 19056 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.702656 19056 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 2 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:52.702991 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:52.703027 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:52.703146 19128 raft_consensus.cc:3055] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:52.703166 19202 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:52.703209 19128 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:52.703406 19128 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.703649 19128 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:52.705046 19202 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 2.
I20251023 20:16:52.705341 19019 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:52.705462 19128 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 2.
I20251023 20:16:52.705519 19841 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:52.705631 19841 raft_consensus.cc:697] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:52.705888 19841 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } }
I20251023 20:16:52.707969 18952 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 1 to 2, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 2 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:52.825064 19056 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 20:16:52.825522 19832 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:52.828611 19128 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 20:16:52.829182 19858 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:52.968663 19202 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251023 20:16:52.969158 19841 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:52.972765 19128 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251023 20:16:52.973363 19872 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:53.119290 19128 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:53.119793 19872 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:53.122877 19202 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:53.123432 19872 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:53.435890 19056 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
W20251023 20:16:53.437913 19020 consensus_peers.cc:597] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7184738166284bf3a71cb7c0d38cd863. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:53.437918 19202 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 20:16:53.437918 19128 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251023 20:16:53.438675 19872 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:53.438891 19821 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:53.440507 19841 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.441022 19128 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.443590 19202 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index -1 to 30, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.444877 19128 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:53.444738 18946 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index -1 to 30, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New cstate: current_term: 2 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.461982 19056 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:53.462046 19202 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251023 20:16:53.462603 19881 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:16:53.462973 19843 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251023 20:16:53.463189 19098 consensus_peers.cc:597] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 -> Peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571): Couldn't send request to peer 3c36454265a141c0adf91706fa3115e9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37f44fa5d63049edbd7ad28d358075fb. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:53.470319 19056 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.469856 19881 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.474184 18947 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New cstate: current_term: 2 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.476789 19203 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:53.478914 19056 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
W20251023 20:16:53.478919 19170 consensus_peers.cc:597] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c0d82f5ba4a476583099f65588d84f0. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:53.479470 19128 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:53.479619 19858 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:53.480657 19858 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:53.482416 19885 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index -1 to 32, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.483835 19056 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index -1 to 32, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.487372 19128 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index -1 to 32, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.490365 18947 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 32, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.489564 19202 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index -1 to 29, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.492810 19598 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:53.501659 19056 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:53.502077 19202 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:53.502086 19799 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:53.502513 19888 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:53.506067 19799 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.506642 19202 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.510722 19056 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 4.30, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:53.510780 18952 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New cstate: current_term: 2 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:53.513089 19202 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
W20251023 20:16:53.513180 19020 consensus_peers.cc:597] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 -> Peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839): Couldn't send request to peer 4cccadfc20fb475c9e6c1ce6cd3ba850. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc60b796f0d946fbb49d15812e88ac60. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:53.513701 19821 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:53.513113 19128 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 20:16:53.515604 19821 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:53.517014 19841 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 LEADER]: Committing config change with OpId 4.31: config changed from index -1 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:53.517560 19202 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index -1 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:53.518909 19128 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index -1 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:53.522697 18946 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index -1 to 31, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New cstate: current_term: 4 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:16:53.524502 19565 consensus_peers.cc:597] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 -> Peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571): Couldn't send request to peer 3c36454265a141c0adf91706fa3115e9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37a00c71a3934405917c05725c94dde5. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:53.526954 19056 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:53.534205 19448 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:53.536247 19056 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:53.536808 19897 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:53.536932 19128 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251023 20:16:53.537441 19749 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:53.540122 19897 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.542256 19056 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
W20251023 20:16:53.545569 19413 consensus_peers.cc:597] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4ff9ca441b14a468709c0efd0ae984e. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:53.545933 18947 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 31 to 33, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New cstate: current_term: 2 leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:53.554911 19128 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } }
I20251023 20:16:53.835134 19903 ts_tablet_manager.cc:933] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: Initiating tablet copy from peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:53.835698 19903 tablet_copy_client.cc:323] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:36987
I20251023 20:16:53.836443 19067 tablet_copy_service.cc:140] P 116b80452edd43bc9d822d31be0c5c38: Received BeginTabletCopySession request for tablet 7184738166284bf3a71cb7c0d38cd863 from peer 329b90f727b745ddacc774c89c3a4428 ({username='slave'} at 127.0.0.1:49630)
I20251023 20:16:53.836558 19067 tablet_copy_service.cc:161] P 116b80452edd43bc9d822d31be0c5c38: Beginning new tablet copy session on tablet 7184738166284bf3a71cb7c0d38cd863 from peer 329b90f727b745ddacc774c89c3a4428 at {username='slave'} at 127.0.0.1:49630: session id = 329b90f727b745ddacc774c89c3a4428-7184738166284bf3a71cb7c0d38cd863
I20251023 20:16:53.838634 19067 tablet_copy_source_session.cc:215] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:53.839618 19903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7184738166284bf3a71cb7c0d38cd863. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:53.844202 19903 tablet_copy_client.cc:806] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:53.844408 19903 tablet_copy_client.cc:670] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:53.845686 19904 tablet_copy_client.cc:1025] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:53.923134 19905 ts_tablet_manager.cc:933] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: Initiating tablet copy from peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:16:53.923691 19905 tablet_copy_client.cc:323] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.198:43569
I20251023 20:16:53.928467 19609 tablet_copy_service.cc:140] P 329b90f727b745ddacc774c89c3a4428: Received BeginTabletCopySession request for tablet 37a00c71a3934405917c05725c94dde5 from peer 3c36454265a141c0adf91706fa3115e9 ({username='slave'} at 127.0.0.1:40606)
I20251023 20:16:53.928592 19609 tablet_copy_service.cc:161] P 329b90f727b745ddacc774c89c3a4428: Beginning new tablet copy session on tablet 37a00c71a3934405917c05725c94dde5 from peer 3c36454265a141c0adf91706fa3115e9 at {username='slave'} at 127.0.0.1:40606: session id = 3c36454265a141c0adf91706fa3115e9-37a00c71a3934405917c05725c94dde5
I20251023 20:16:53.930589 19609 tablet_copy_source_session.cc:215] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:53.931744 19905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a00c71a3934405917c05725c94dde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:53.936288 19905 tablet_copy_client.cc:806] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:53.936532 19905 tablet_copy_client.cc:670] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:53.937822 19906 tablet_copy_client.cc:1025] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:53.956671 19908 ts_tablet_manager.cc:933] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: Initiating tablet copy from peer f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:53.957260 19908 tablet_copy_client.cc:323] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:33259
I20251023 20:16:53.957762 19142 tablet_copy_service.cc:140] P f909677d13e94318ad9a3b1b2db05723: Received BeginTabletCopySession request for tablet 37f44fa5d63049edbd7ad28d358075fb from peer 3c36454265a141c0adf91706fa3115e9 ({username='slave'} at 127.0.0.1:49234)
I20251023 20:16:53.957887 19142 tablet_copy_service.cc:161] P f909677d13e94318ad9a3b1b2db05723: Beginning new tablet copy session on tablet 37f44fa5d63049edbd7ad28d358075fb from peer 3c36454265a141c0adf91706fa3115e9 at {username='slave'} at 127.0.0.1:49234: session id = 3c36454265a141c0adf91706fa3115e9-37f44fa5d63049edbd7ad28d358075fb
I20251023 20:16:53.960330 19142 tablet_copy_source_session.cc:215] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:53.961573 19908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44fa5d63049edbd7ad28d358075fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:53.966413 19908 tablet_copy_client.cc:806] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:53.966615 19908 tablet_copy_client.cc:670] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:53.967911 19909 tablet_copy_client.cc:1025] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:54.040727 19910 ts_tablet_manager.cc:933] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Initiating tablet copy from peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:54.041339 19910 tablet_copy_client.cc:323] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:36987
I20251023 20:16:54.042060 19067 tablet_copy_service.cc:140] P 116b80452edd43bc9d822d31be0c5c38: Received BeginTabletCopySession request for tablet fc60b796f0d946fbb49d15812e88ac60 from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 ({username='slave'} at 127.0.0.1:55958)
I20251023 20:16:54.042201 19067 tablet_copy_service.cc:161] P 116b80452edd43bc9d822d31be0c5c38: Beginning new tablet copy session on tablet fc60b796f0d946fbb49d15812e88ac60 from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 at {username='slave'} at 127.0.0.1:55958: session id = 4cccadfc20fb475c9e6c1ce6cd3ba850-fc60b796f0d946fbb49d15812e88ac60
I20251023 20:16:54.044595 19067 tablet_copy_source_session.cc:215] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:54.045516 19910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc60b796f0d946fbb49d15812e88ac60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:54.050400 19910 tablet_copy_client.cc:806] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:54.050614 19910 tablet_copy_client.cc:670] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:54.051648 19911 tablet_copy_client.cc:1025] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:54.090030 19913 ts_tablet_manager.cc:933] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: Initiating tablet copy from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:16:54.090588 19913 tablet_copy_client.cc:323] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.196:34839
I20251023 20:16:54.096282 19459 tablet_copy_service.cc:140] P 4cccadfc20fb475c9e6c1ce6cd3ba850: Received BeginTabletCopySession request for tablet f4ff9ca441b14a468709c0efd0ae984e from peer 329b90f727b745ddacc774c89c3a4428 ({username='slave'} at 127.0.0.1:35700)
I20251023 20:16:54.096406 19459 tablet_copy_service.cc:161] P 4cccadfc20fb475c9e6c1ce6cd3ba850: Beginning new tablet copy session on tablet f4ff9ca441b14a468709c0efd0ae984e from peer 329b90f727b745ddacc774c89c3a4428 at {username='slave'} at 127.0.0.1:35700: session id = 329b90f727b745ddacc774c89c3a4428-f4ff9ca441b14a468709c0efd0ae984e
I20251023 20:16:54.098632 19459 tablet_copy_source_session.cc:215] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:54.099836 19913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ff9ca441b14a468709c0efd0ae984e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:54.100383 19917 ts_tablet_manager.cc:933] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: Initiating tablet copy from peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:54.101094 19917 tablet_copy_client.cc:323] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:37799
I20251023 20:16:54.101680 19216 tablet_copy_service.cc:140] P cea0984de40845da94e6f23513009f02: Received BeginTabletCopySession request for tablet 7c0d82f5ba4a476583099f65588d84f0 from peer 329b90f727b745ddacc774c89c3a4428 ({username='slave'} at 127.0.0.1:58470)
I20251023 20:16:54.101801 19216 tablet_copy_service.cc:161] P cea0984de40845da94e6f23513009f02: Beginning new tablet copy session on tablet 7c0d82f5ba4a476583099f65588d84f0 from peer 329b90f727b745ddacc774c89c3a4428 at {username='slave'} at 127.0.0.1:58470: session id = 329b90f727b745ddacc774c89c3a4428-7c0d82f5ba4a476583099f65588d84f0
I20251023 20:16:54.104324 19216 tablet_copy_source_session.cc:215] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:54.105255 19917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0d82f5ba4a476583099f65588d84f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:54.105775 19913 tablet_copy_client.cc:806] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:54.105978 19913 tablet_copy_client.cc:670] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:54.107427 19914 tablet_copy_client.cc:1025] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:54.110532 19917 tablet_copy_client.cc:806] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:54.110738 19917 tablet_copy_client.cc:670] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:54.111876 19918 tablet_copy_client.cc:1025] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:54.148301 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:54.151546 19203 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
new_leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
from {username='slave'} at 127.0.0.1:53226
I20251023 20:16:54.151731 19203 raft_consensus.cc:606] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Received request to transfer leadership to TS 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:54.152593 19054 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:54.152731 19054 raft_consensus.cc:606] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:54.153614 19054 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:16:54.153728 19054 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Received request to transfer leadership to TS cea0984de40845da94e6f23513009f02
I20251023 20:16:54.154038 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251023 20:16:54.154196 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:54.191668 19885 raft_consensus.cc:993] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: : Instructing follower 4cccadfc20fb475c9e6c1ce6cd3ba850 to start an election
I20251023 20:16:54.191815 19885 raft_consensus.cc:1081] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Signalling peer 4cccadfc20fb475c9e6c1ce6cd3ba850 to start an election
I20251023 20:16:54.192337 19449 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831"
dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
from {username='slave'} at 127.0.0.1:48970
I20251023 20:16:54.192484 19449 raft_consensus.cc:493] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:54.192576 19449 raft_consensus.cc:3060] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:54.194938 19449 raft_consensus.cc:515] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:54.195729 19449 leader_election.cc:290] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [CANDIDATE]: Term 2 election: Requested vote from peers cea0984de40845da94e6f23513009f02 (127.15.81.195:37799), 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571)
I20251023 20:16:54.196022 19203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831" candidate_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:54.196161 19524 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "461a7ab7c21c40fbb4ef24809dad5831" candidate_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "3c36454265a141c0adf91706fa3115e9"
I20251023 20:16:54.196187 19203 raft_consensus.cc:3055] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:16:54.196390 19524 raft_consensus.cc:3060] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:54.196411 19203 raft_consensus.cc:740] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:54.196678 19203 consensus_queue.cc:260] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:54.196961 19203 raft_consensus.cc:3060] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:16:54.198275 19524 raft_consensus.cc:2468] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cccadfc20fb475c9e6c1ce6cd3ba850 in term 2.
I20251023 20:16:54.198625 19415 leader_election.cc:304] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c36454265a141c0adf91706fa3115e9, 4cccadfc20fb475c9e6c1ce6cd3ba850; no voters:
I20251023 20:16:54.198683 19203 raft_consensus.cc:2468] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cccadfc20fb475c9e6c1ce6cd3ba850 in term 2.
I20251023 20:16:54.198850 19902 raft_consensus.cc:2804] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:16:54.199265 19902 raft_consensus.cc:697] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Becoming Leader. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Running, Role: LEADER
I20251023 20:16:54.199491 19902 consensus_queue.cc:237] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:54.201659 18952 catalog_manager.cc:5649] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: term changed from 1 to 2, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196). New cstate: current_term: 2 leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:54.467763 19928 raft_consensus.cc:993] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:54.467921 19928 raft_consensus.cc:1081] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Signalling peer cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:54.468361 19203 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:53202
I20251023 20:16:54.468509 19203 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:54.468592 19203 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:54.470266 19203 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:54.470857 19203 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 3 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:54.471139 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:54.471036 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:54.471267 19131 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:54.471386 19056 raft_consensus.cc:3055] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:16:54.471448 19056 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:54.471662 19056 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:54.471969 19056 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:54.473809 19131 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 3.
I20251023 20:16:54.474159 19170 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:54.474400 19885 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:16:54.474512 19885 raft_consensus.cc:697] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:54.474705 19885 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:54.474710 19056 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 3.
I20251023 20:16:54.477452 18952 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 2 to 3, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 3 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:54.478094 19928 raft_consensus.cc:993] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:54.478195 19821 raft_consensus.cc:1081] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:54.478789 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:16:54.478899 19128 raft_consensus.cc:493] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:54.478977 19128 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:54.480835 19128 raft_consensus.cc:515] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:54.481307 19128 leader_election.cc:290] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 3 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571)
I20251023 20:16:54.481732 19524 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "3c36454265a141c0adf91706fa3115e9"
I20251023 20:16:54.481732 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "508756851fc14f93a7d310b11beb15cb" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 3 candidate_status { last_received { term: 2 index: 32 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:54.481915 19056 raft_consensus.cc:3055] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:16:54.481915 19524 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:54.482002 19056 raft_consensus.cc:740] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:54.482177 19056 consensus_queue.cc:260] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:54.482409 19056 raft_consensus.cc:3060] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:54.483620 19524 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 3.
I20251023 20:16:54.483899 19098 leader_election.cc:304] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c36454265a141c0adf91706fa3115e9, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:54.483943 19056 raft_consensus.cc:2468] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 3.
I20251023 20:16:54.484107 19843 raft_consensus.cc:2804] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:16:54.484514 19843 raft_consensus.cc:697] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:54.484712 19843 consensus_queue.cc:237] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 32, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:54.486764 18952 catalog_manager.cc:5649] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 2 to 3, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 3 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:54.489475 19203 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:54.489600 19203 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Received request to transfer leadership
I20251023 20:16:54.495575 19202 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:54.495682 19202 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Received request to transfer leadership
W20251023 20:16:54.495975 18938 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 7184738166284bf3a71cb7c0d38cd863 already in progress [suppressed 1 similar messages]
I20251023 20:16:54.496791 19202 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:54.496937 19202 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Received request to transfer leadership
W20251023 20:16:54.497293 19002 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 7184738166284bf3a71cb7c0d38cd863 already in progress
W20251023 20:16:54.497352 19002 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 7184738166284bf3a71cb7c0d38cd863 already in progress
I20251023 20:16:54.634243 19524 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:54.637104 19935 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:54.641085 19202 raft_consensus.cc:1275] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:54.641993 19935 consensus_queue.cc:1048] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:54.915657 19524 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 20:16:54.916139 19930 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:54.919724 19054 raft_consensus.cc:1275] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 20:16:54.920456 19843 consensus_queue.cc:1048] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:55.002715 19128 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 20:16:55.003173 19885 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:55.006206 19885 raft_consensus.cc:993] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:55.006315 19885 raft_consensus.cc:1081] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:55.007651 19131 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36232
I20251023 20:16:55.007792 19131 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:55.007872 19131 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:55.007870 19054 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 20:16:55.008338 19919 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
W20251023 20:16:55.010612 19170 consensus_peers.cc:597] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:55.011312 19131 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:55.012104 19131 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 4 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:55.012451 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:55.012291 19128 raft_consensus.cc:1240] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Rejecting Update request from peer cea0984de40845da94e6f23513009f02 for earlier term 3. Current term is 4. Ops: []
I20251023 20:16:55.013125 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:55.013283 19202 raft_consensus.cc:3055] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 20:16:55.013324 19919 consensus_queue.cc:1059] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: INVALID_TERM, Last received: 3.31, Next index: 32, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:55.013340 19202 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:55.013911 19202 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:55.014209 19202 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:55.016170 19202 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 4.
W20251023 20:16:55.016310 19934 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20251023 20:16:55.016556 19095 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cea0984de40845da94e6f23513009f02, f909677d13e94318ad9a3b1b2db05723; no voters: 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:55.017366 19933 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 20:16:55.017493 19933 raft_consensus.cc:697] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:55.017688 19933 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:55.020220 18952 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 3 to 4, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 4 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:55.447466 19202 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:55.447984 19933 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
W20251023 20:16:55.452919 19096 consensus_peers.cc:597] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 -> Peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569): Couldn't send request to peer 329b90f727b745ddacc774c89c3a4428. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:55.453577 19056 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:55.456077 19056 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:55.456550 19933 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:55.846022 19903 tablet_copy_client.cc:538] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:55.849151 19903 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: Bootstrap starting.
I20251023 20:16:55.884310 19903 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=104 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:55.884727 19903 tablet_bootstrap.cc:492] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: Bootstrap complete.
I20251023 20:16:55.885106 19903 ts_tablet_manager.cc:1403] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: Time spent bootstrapping tablet: real 0.036s user 0.027s sys 0.009s
I20251023 20:16:55.885821 19903 raft_consensus.cc:359] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:55.886025 19903 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Initialized, Role: LEARNER
I20251023 20:16:55.886263 19903 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:55.886690 19903 ts_tablet_manager.cc:1434] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:16:55.887221 19067 tablet_copy_service.cc:342] P 116b80452edd43bc9d822d31be0c5c38: Request end of tablet copy session 329b90f727b745ddacc774c89c3a4428-7184738166284bf3a71cb7c0d38cd863 received from {username='slave'} at 127.0.0.1:49630
I20251023 20:16:55.887352 19067 tablet_copy_service.cc:434] P 116b80452edd43bc9d822d31be0c5c38: ending tablet copy session 329b90f727b745ddacc774c89c3a4428-7184738166284bf3a71cb7c0d38cd863 on tablet 7184738166284bf3a71cb7c0d38cd863 with peer 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:55.938189 19905 tablet_copy_client.cc:538] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:55.940937 19905 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: Bootstrap starting.
I20251023 20:16:55.968304 19908 tablet_copy_client.cc:538] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:55.971647 19908 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: Bootstrap starting.
I20251023 20:16:55.983201 19905 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=120 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:55.983559 19905 tablet_bootstrap.cc:492] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: Bootstrap complete.
I20251023 20:16:55.983860 19905 ts_tablet_manager.cc:1403] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: Time spent bootstrapping tablet: real 0.043s user 0.038s sys 0.004s
I20251023 20:16:55.984711 19905 raft_consensus.cc:359] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:55.985044 19905 raft_consensus.cc:740] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Initialized, Role: LEARNER
I20251023 20:16:55.985350 19905 consensus_queue.cc:260] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:55.985875 19905 ts_tablet_manager.cc:1434] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:16:55.987138 19609 tablet_copy_service.cc:342] P 329b90f727b745ddacc774c89c3a4428: Request end of tablet copy session 3c36454265a141c0adf91706fa3115e9-37a00c71a3934405917c05725c94dde5 received from {username='slave'} at 127.0.0.1:40606
I20251023 20:16:55.987260 19609 tablet_copy_service.cc:434] P 329b90f727b745ddacc774c89c3a4428: ending tablet copy session 3c36454265a141c0adf91706fa3115e9-37a00c71a3934405917c05725c94dde5 on tablet 37a00c71a3934405917c05725c94dde5 with peer 3c36454265a141c0adf91706fa3115e9
I20251023 20:16:56.007417 19908 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=87 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:56.007777 19908 tablet_bootstrap.cc:492] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: Bootstrap complete.
I20251023 20:16:56.008018 19908 ts_tablet_manager.cc:1403] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: Time spent bootstrapping tablet: real 0.036s user 0.028s sys 0.007s
I20251023 20:16:56.008620 19908 raft_consensus.cc:359] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.008769 19908 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Initialized, Role: LEARNER
I20251023 20:16:56.008961 19908 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.009425 19908 ts_tablet_manager.cc:1434] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20251023 20:16:56.009995 19142 tablet_copy_service.cc:342] P f909677d13e94318ad9a3b1b2db05723: Request end of tablet copy session 3c36454265a141c0adf91706fa3115e9-37f44fa5d63049edbd7ad28d358075fb received from {username='slave'} at 127.0.0.1:49234
I20251023 20:16:56.010128 19142 tablet_copy_service.cc:434] P f909677d13e94318ad9a3b1b2db05723: ending tablet copy session 3c36454265a141c0adf91706fa3115e9-37f44fa5d63049edbd7ad28d358075fb on tablet 37f44fa5d63049edbd7ad28d358075fb with peer 3c36454265a141c0adf91706fa3115e9
I20251023 20:16:56.051957 19910 tablet_copy_client.cc:538] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:56.054780 19910 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap starting.
I20251023 20:16:56.058548 19599 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Advancing to term 4
I20251023 20:16:56.060822 19599 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 4 LEARNER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251023 20:16:56.061388 19933 consensus_queue.cc:1050] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:56.065241 19933 raft_consensus.cc:1064] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: attempting to promote NON_VOTER 329b90f727b745ddacc774c89c3a4428 to VOTER
I20251023 20:16:56.065950 19933 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 31, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.068296 19202 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.068830 19056 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.068795 19933 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.069258 19930 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.069572 19599 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 4 LEARNER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.070799 19930 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.071863 19930 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 30 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.075611 19203 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 30 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.075702 19599 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 30 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.080353 18947 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 30 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.082193 19054 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 30 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.092993 19131 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.094352 19599 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 20:16:56.094576 19054 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 20:16:56.094890 19961 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.095161 19930 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.097357 19930 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.097890 19599 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.098853 19910 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:56.099200 19910 tablet_bootstrap.cc:492] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap complete.
I20251023 20:16:56.099432 19910 ts_tablet_manager.cc:1403] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent bootstrapping tablet: real 0.045s user 0.041s sys 0.000s
I20251023 20:16:56.100157 19910 raft_consensus.cc:359] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:56.100031 19054 raft_consensus.cc:2955] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.100337 19910 raft_consensus.cc:740] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Initialized, Role: LEARNER
I20251023 20:16:56.100502 19910 consensus_queue.cc:260] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 4.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:56.101157 19910 ts_tablet_manager.cc:1434] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251023 20:16:56.101065 18951 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 33 to 34, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New cstate: current_term: 4 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:56.100498 18939 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 7184738166284bf3a71cb7c0d38cd863 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:56.103072 19067 tablet_copy_service.cc:342] P 116b80452edd43bc9d822d31be0c5c38: Request end of tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-fc60b796f0d946fbb49d15812e88ac60 received from {username='slave'} at 127.0.0.1:55958
I20251023 20:16:56.103165 19067 tablet_copy_service.cc:434] P 116b80452edd43bc9d822d31be0c5c38: ending tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-fc60b796f0d946fbb49d15812e88ac60 on tablet fc60b796f0d946fbb49d15812e88ac60 with peer 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:16:56.105281 19196 tablet_service.cc:1552] Processing DeleteTablet for tablet 7184738166284bf3a71cb7c0d38cd863 with delete_type TABLET_DATA_TOMBSTONED (TS cea0984de40845da94e6f23513009f02 not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:56.105814 19966 tablet_replica.cc:333] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:16:56.106314 19966 raft_consensus.cc:2243] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:56.106729 19966 raft_consensus.cc:2272] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:56.107967 19966 ts_tablet_manager.cc:1916] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:56.108441 19913 tablet_copy_client.cc:538] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:56.111362 19913 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: Bootstrap starting.
I20251023 20:16:56.112169 19917 tablet_copy_client.cc:538] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:16:56.113417 19966 ts_tablet_manager.cc:1929] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.33
I20251023 20:16:56.113560 19966 log.cc:1199] T 7184738166284bf3a71cb7c0d38cd863 P cea0984de40845da94e6f23513009f02: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/7184738166284bf3a71cb7c0d38cd863
I20251023 20:16:56.116422 18938 catalog_manager.cc:4985] TS cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): tablet 7184738166284bf3a71cb7c0d38cd863 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:56.116621 19917 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: Bootstrap starting.
I20251023 20:16:56.154512 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:56.156991 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:16:56.157228 19128 raft_consensus.cc:606] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:16:56.158164 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "329b90f727b745ddacc774c89c3a4428"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:16:56.158350 19128 raft_consensus.cc:606] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Received request to transfer leadership to TS 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:56.158707 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251023 20:16:56.158917 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:56.160445 19913 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:56.160895 19913 tablet_bootstrap.cc:492] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: Bootstrap complete.
I20251023 20:16:56.161235 19913 ts_tablet_manager.cc:1403] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: Time spent bootstrapping tablet: real 0.050s user 0.044s sys 0.005s
I20251023 20:16:56.162081 19913 raft_consensus.cc:359] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:56.162308 19913 raft_consensus.cc:740] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Initialized, Role: LEARNER
I20251023 20:16:56.162375 19917 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=132 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:16:56.162528 19913 consensus_queue.cc:260] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:56.162698 19917 tablet_bootstrap.cc:492] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: Bootstrap complete.
I20251023 20:16:56.162922 19917 ts_tablet_manager.cc:1403] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: Time spent bootstrapping tablet: real 0.046s user 0.042s sys 0.004s
I20251023 20:16:56.163790 19913 ts_tablet_manager.cc:1434] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:16:56.163801 19917 raft_consensus.cc:359] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:56.165514 19917 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Initialized, Role: LEARNER
I20251023 20:16:56.165728 19917 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } }
I20251023 20:16:56.166134 19917 ts_tablet_manager.cc:1434] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251023 20:16:56.166438 19459 tablet_copy_service.cc:342] P 4cccadfc20fb475c9e6c1ce6cd3ba850: Request end of tablet copy session 329b90f727b745ddacc774c89c3a4428-f4ff9ca441b14a468709c0efd0ae984e received from {username='slave'} at 127.0.0.1:35700
I20251023 20:16:56.166580 19459 tablet_copy_service.cc:434] P 4cccadfc20fb475c9e6c1ce6cd3ba850: ending tablet copy session 329b90f727b745ddacc774c89c3a4428-f4ff9ca441b14a468709c0efd0ae984e on tablet f4ff9ca441b14a468709c0efd0ae984e with peer 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:56.166843 19216 tablet_copy_service.cc:342] P cea0984de40845da94e6f23513009f02: Request end of tablet copy session 329b90f727b745ddacc774c89c3a4428-7c0d82f5ba4a476583099f65588d84f0 received from {username='slave'} at 127.0.0.1:58470
I20251023 20:16:56.166963 19216 tablet_copy_service.cc:434] P cea0984de40845da94e6f23513009f02: ending tablet copy session 329b90f727b745ddacc774c89c3a4428-7c0d82f5ba4a476583099f65588d84f0 on tablet 7c0d82f5ba4a476583099f65588d84f0 with peer 329b90f727b745ddacc774c89c3a4428
I20251023 20:16:56.181799 19524 raft_consensus.cc:1217] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.32->[2.33-2.33] Dedup: 2.33->[]
I20251023 20:16:56.191766 19930 raft_consensus.cc:993] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:56.191919 19961 raft_consensus.cc:1081] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:16:56.192466 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:16:56.192657 19056 raft_consensus.cc:493] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:56.192755 19056 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:56.194797 19056 raft_consensus.cc:515] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.195513 19056 leader_election.cc:290] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 3 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), cea0984de40845da94e6f23513009f02 (127.15.81.195:37799)
I20251023 20:16:56.195683 19203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:56.195839 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:56.195868 19203 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:56.196026 19128 raft_consensus.cc:3055] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:16:56.196116 19128 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:56.196358 19128 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.196759 19128 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:56.197976 19203 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 3.
I20251023 20:16:56.198374 19019 leader_election.cc:304] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:16:56.198681 19967 raft_consensus.cc:2804] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:16:56.198841 19967 raft_consensus.cc:697] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:56.198838 19128 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 3.
I20251023 20:16:56.199100 19967 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.201644 18952 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 2 to 3, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 3 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:56.209570 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:56.209717 19056 raft_consensus.cc:606] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Received request to transfer leadership
I20251023 20:16:56.214689 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:56.214828 19056 raft_consensus.cc:606] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Received request to transfer leadership
I20251023 20:16:56.215811 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:56.215945 19056 raft_consensus.cc:606] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Received request to transfer leadership
W20251023 20:16:56.216369 19002 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 37f44fa5d63049edbd7ad28d358075fb already in progress
W20251023 20:16:56.216465 19002 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 37f44fa5d63049edbd7ad28d358075fb already in progress
I20251023 20:16:56.238293 19599 raft_consensus.cc:1217] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.31->[4.32-4.32] Dedup: 4.32->[]
I20251023 20:16:56.464283 19449 raft_consensus.cc:1217] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.30->[4.31-4.31] Dedup: 4.31->[]
I20251023 20:16:56.569427 19930 raft_consensus.cc:993] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 329b90f727b745ddacc774c89c3a4428 to start an election
I20251023 20:16:56.569600 19961 raft_consensus.cc:1081] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Signalling peer 329b90f727b745ddacc774c89c3a4428 to start an election
I20251023 20:16:56.570118 19599 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11"
dest_uuid: "329b90f727b745ddacc774c89c3a4428"
from {username='slave'} at 127.0.0.1:40588
I20251023 20:16:56.570322 19599 raft_consensus.cc:493] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:56.570425 19599 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:56.572726 19599 raft_consensus.cc:515] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.573379 19599 leader_election.cc:290] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [CANDIDATE]: Term 3 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:56.573647 19054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "329b90f727b745ddacc774c89c3a4428" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:56.573801 19054 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:56.575479 19054 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329b90f727b745ddacc774c89c3a4428 in term 3.
I20251023 20:16:56.575796 19565 leader_election.cc:304] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, 329b90f727b745ddacc774c89c3a4428; no voters:
I20251023 20:16:56.576031 19960 raft_consensus.cc:2804] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:16:56.576434 19960 raft_consensus.cc:697] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 LEADER]: Becoming Leader. State: Replica: 329b90f727b745ddacc774c89c3a4428, State: Running, Role: LEADER
I20251023 20:16:56.576684 19960 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.579452 18952 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: term changed from 2 to 3, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 329b90f727b745ddacc774c89c3a4428 (127.15.81.198). New cstate: current_term: 3 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:56.583871 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38153cd2d04e4c56b22f87e07a3c2d11" candidate_uuid: "329b90f727b745ddacc774c89c3a4428" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:56.584022 19128 raft_consensus.cc:3055] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:16:56.584082 19128 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:56.584288 19128 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.584565 19128 raft_consensus.cc:3060] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:16:56.586575 19128 raft_consensus.cc:2468] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329b90f727b745ddacc774c89c3a4428 in term 3.
I20251023 20:16:56.594281 19974 raft_consensus.cc:1064] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: attempting to promote NON_VOTER 3c36454265a141c0adf91706fa3115e9 to VOTER
I20251023 20:16:56.595026 19974 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 31, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:56.597260 19054 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:56.597394 19524 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:56.597817 19976 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.598296 19203 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:56.598692 19976 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.599761 19960 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.609675 19978 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.610256 19054 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.614127 18952 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.614151 19203 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.615124 19524 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.622880 19599 consensus_queue.cc:237] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:56.624508 19524 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251023 20:16:56.624912 19976 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:16:56.625007 19054 raft_consensus.cc:1275] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251023 20:16:56.625360 19976 consensus_queue.cc:1048] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:16:56.627161 19974 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.627816 19054 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.630020 18998 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 37a00c71a3934405917c05725c94dde5 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:56.630165 19524 raft_consensus.cc:2955] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:56.631860 18952 catalog_manager.cc:5649] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 34 to 35, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New cstate: current_term: 2 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:56.635640 19196 tablet_service.cc:1552] Processing DeleteTablet for tablet 37a00c71a3934405917c05725c94dde5 with delete_type TABLET_DATA_TOMBSTONED (TS cea0984de40845da94e6f23513009f02 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:53196
I20251023 20:16:56.636130 19982 tablet_replica.cc:333] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:16:56.637197 19982 raft_consensus.cc:2243] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:56.637527 19982 raft_consensus.cc:2272] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:56.638249 19982 ts_tablet_manager.cc:1916] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:56.642377 19982 ts_tablet_manager.cc:1929] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.34
I20251023 20:16:56.642496 19982 log.cc:1199] T 37a00c71a3934405917c05725c94dde5 P cea0984de40845da94e6f23513009f02: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/37a00c71a3934405917c05725c94dde5
I20251023 20:16:56.643038 18938 catalog_manager.cc:4985] TS cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): tablet 37a00c71a3934405917c05725c94dde5 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:56.649654 19202 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 20:16:56.650065 19967 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:56.651898 19952 raft_consensus.cc:993] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:56.652022 19967 raft_consensus.cc:1081] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Signalling peer cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:16:56.652510 19203 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb"
dest_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:53202
I20251023 20:16:56.657871 19203 raft_consensus.cc:493] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:56.657987 19203 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:56.658414 19128 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 20:16:56.658850 19967 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:56.660185 19203 raft_consensus.cc:515] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.661190 19524 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 2 LEARNER]: Advancing to term 3
I20251023 20:16:56.662106 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:16:56.662556 19203 leader_election.cc:290] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 4 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259)
I20251023 20:16:56.662814 19054 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37f44fa5d63049edbd7ad28d358075fb" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:16:56.662982 19054 raft_consensus.cc:3055] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 20:16:56.663069 19054 raft_consensus.cc:740] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:16:56.663355 19054 consensus_queue.cc:260] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.663532 19524 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 3 LEARNER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251023 20:16:56.663703 19054 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:56.665803 19054 raft_consensus.cc:2468] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 4.
I20251023 20:16:56.666234 19172 leader_election.cc:304] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, cea0984de40845da94e6f23513009f02; no voters: f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:56.666462 19885 raft_consensus.cc:2804] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 20:16:56.666850 19885 raft_consensus.cc:697] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:56.667112 19885 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:56.669180 18946 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 3 to 4, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:56.737092 19599 raft_consensus.cc:1217] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.32->[2.33-2.33] Dedup: 2.33->[]
I20251023 20:16:56.847587 19990 raft_consensus.cc:1064] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 329b90f727b745ddacc774c89c3a4428 to VOTER
I20251023 20:16:56.848413 19990 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 30, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.850683 19599 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.850716 19131 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.851229 19990 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.851313 19054 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.851722 19990 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.852799 19885 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.855093 19990 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.855659 19131 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.859326 19599 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.861189 18946 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 32 to 33, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.864955 19984 raft_consensus.cc:1064] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: attempting to promote NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 to VOTER
I20251023 20:16:56.865531 19984 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 29, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:56.867940 19202 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:56.868531 19203 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:56.869053 19131 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:56.869689 19967 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:56.870131 19967 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:56.871501 19449 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 LEARNER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:56.872174 19997 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:56.872709 19599 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 20:16:56.873040 19131 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251023 20:16:56.873224 19990 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.874574 19990 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:56.876009 19989 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.877400 19997 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.878180 19128 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.880460 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 7c0d82f5ba4a476583099f65588d84f0 with cas_config_opid_index 33: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:56.881631 18952 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 33 to 34, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:56.882386 19131 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.883013 19202 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.883482 18951 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:56.885471 19599 raft_consensus.cc:2955] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 33 to 34, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:56.888613 19041 tablet_service.cc:1552] Processing DeleteTablet for tablet 7c0d82f5ba4a476583099f65588d84f0 with delete_type TABLET_DATA_TOMBSTONED (TS 116b80452edd43bc9d822d31be0c5c38 not found in new config with opid_index 34) from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:56.893285 20001 tablet_replica.cc:333] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:16:56.893479 20001 raft_consensus.cc:2243] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:56.893625 20001 pending_rounds.cc:70] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Trying to abort 1 pending ops.
I20251023 20:16:56.893694 20001 pending_rounds.cc:77] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Aborting op as it isn't in flight: id { term: 4 index: 33 } timestamp: 7214082526609731584 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "7c0d82f5ba4a476583099f65588d84f0" old_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } } new_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } } }
I20251023 20:16:56.894006 20001 raft_consensus.cc:2917] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Aborting config change with OpId 4.33: Aborted: Op aborted
I20251023 20:16:56.894141 20001 raft_consensus.cc:2889] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20251023 20:16:56.894218 20001 raft_consensus.cc:2272] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:56.895051 20001 ts_tablet_manager.cc:1916] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:56.899916 20001 ts_tablet_manager.cc:1929] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.33
I20251023 20:16:56.900585 20001 log.cc:1199] T 7c0d82f5ba4a476583099f65588d84f0 P 116b80452edd43bc9d822d31be0c5c38: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/7c0d82f5ba4a476583099f65588d84f0
I20251023 20:16:56.898965 19448 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.901515 19056 consensus_queue.cc:237] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:16:56.902982 18941 catalog_manager.cc:4985] TS 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): tablet 7c0d82f5ba4a476583099f65588d84f0 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:56.903524 19449 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.904508 19967 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.905112 19202 raft_consensus.cc:1275] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:56.905562 19967 consensus_queue.cc:1048] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:56.906526 19984 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.907346 19449 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.908092 19202 raft_consensus.cc:2955] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.911453 18947 catalog_manager.cc:5649] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 32 to 33, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New cstate: current_term: 4 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:16:56.912240 18941 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet fc60b796f0d946fbb49d15812e88ac60 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:56.914405 19113 tablet_service.cc:1552] Processing DeleteTablet for tablet fc60b796f0d946fbb49d15812e88ac60 with delete_type TABLET_DATA_TOMBSTONED (TS f909677d13e94318ad9a3b1b2db05723 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:36224
I20251023 20:16:56.915551 20002 tablet_replica.cc:333] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:16:56.915756 20002 raft_consensus.cc:2243] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:56.915944 20002 raft_consensus.cc:2272] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:56.917668 20002 ts_tablet_manager.cc:1916] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:56.923226 20002 ts_tablet_manager.cc:1929] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.32
I20251023 20:16:56.923352 20002 log.cc:1199] T fc60b796f0d946fbb49d15812e88ac60 P f909677d13e94318ad9a3b1b2db05723: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/fc60b796f0d946fbb49d15812e88ac60
I20251023 20:16:56.923980 18939 catalog_manager.cc:4985] TS f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): tablet fc60b796f0d946fbb49d15812e88ac60 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:57.163342 19131 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 20:16:57.163823 19978 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:57.171010 19056 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 20:16:57.171797 20004 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:57.231267 19131 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:16:57.233947 19131 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 20:16:57.234421 19992 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:57.241799 19524 raft_consensus.cc:3060] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 3 LEARNER]: Advancing to term 4
I20251023 20:16:57.242115 19056 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 20:16:57.242591 19990 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:16:57.244040 19524 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251023 20:16:57.245970 19990 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:16:57.248617 20012 raft_consensus.cc:1064] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: attempting to promote NON_VOTER 329b90f727b745ddacc774c89c3a4428 to VOTER
I20251023 20:16:57.249378 20012 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 31, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:57.251368 19599 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 LEARNER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:57.251423 19131 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:57.251807 19056 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251023 20:16:57.252012 19936 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:57.252552 19936 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:57.253067 19936 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:16:57.265383 19992 raft_consensus.cc:1064] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: attempting to promote NON_VOTER 3c36454265a141c0adf91706fa3115e9 to VOTER
I20251023 20:16:57.266103 19992 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 30, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:57.267822 19524 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 LEARNER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:57.267887 19131 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:57.268254 19056 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251023 20:16:57.268326 19885 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:57.268692 19885 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:57.269065 19885 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:16:57.270659 20011 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.271167 19885 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index 29 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.271708 19524 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 29 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.275203 19128 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.276449 18952 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 29 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:57.277060 19056 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 29 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.278076 19599 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.279803 18947 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 33 to 34, 329b90f727b745ddacc774c89c3a4428 (127.15.81.198) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.281452 19202 consensus_queue.cc:237] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 30, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:16:57.284175 19131 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 29 to 32, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.284713 19524 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:57.287042 19885 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:57.288494 19448 consensus_queue.cc:237] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:57.288502 19128 raft_consensus.cc:1275] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Refusing update from remote peer cea0984de40845da94e6f23513009f02: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:16:57.290009 19992 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.290479 19989 consensus_queue.cc:1048] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:57.291988 19599 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251023 20:16:57.292472 20015 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:16:57.292589 19128 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.293368 18938 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 37f44fa5d63049edbd7ad28d358075fb with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:57.293749 19131 raft_consensus.cc:1275] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Refusing update from remote peer 4cccadfc20fb475c9e6c1ce6cd3ba850: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251023 20:16:57.293934 19524 raft_consensus.cc:2955] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:16:57.294274 18951 catalog_manager.cc:5649] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index 32 to 33, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New cstate: current_term: 4 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:57.294664 20015 consensus_queue.cc:1048] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:16:57.296370 20011 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.296954 19599 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.297516 19041 tablet_service.cc:1552] Processing DeleteTablet for tablet 37f44fa5d63049edbd7ad28d358075fb with delete_type TABLET_DATA_TOMBSTONED (TS 116b80452edd43bc9d822d31be0c5c38 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:57.297794 20001 tablet_replica.cc:333] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:16:57.297950 20001 raft_consensus.cc:2243] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:57.298123 20001 raft_consensus.cc:2272] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:57.299438 20001 ts_tablet_manager.cc:1916] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:57.299894 18998 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet f4ff9ca441b14a468709c0efd0ae984e with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:16:57.300788 18952 catalog_manager.cc:5649] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 34 to 35, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New cstate: current_term: 2 leader_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:16:57.301363 19131 raft_consensus.cc:2955] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } }
I20251023 20:16:57.304836 19041 tablet_service.cc:1552] Processing DeleteTablet for tablet f4ff9ca441b14a468709c0efd0ae984e with delete_type TABLET_DATA_TOMBSTONED (TS 116b80452edd43bc9d822d31be0c5c38 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:55934
I20251023 20:16:57.306648 20001 ts_tablet_manager.cc:1929] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.32
I20251023 20:16:57.306731 20001 log.cc:1199] T 37f44fa5d63049edbd7ad28d358075fb P 116b80452edd43bc9d822d31be0c5c38: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/37f44fa5d63049edbd7ad28d358075fb
I20251023 20:16:57.307268 20001 tablet_replica.cc:333] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:16:57.307274 18941 catalog_manager.cc:4985] TS 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): tablet 37f44fa5d63049edbd7ad28d358075fb (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:57.307430 20001 raft_consensus.cc:2243] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:16:57.307600 20001 pending_rounds.cc:70] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Trying to abort 1 pending ops.
I20251023 20:16:57.307639 20001 pending_rounds.cc:77] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Aborting op as it isn't in flight: id { term: 2 index: 34 } timestamp: 7214082528251777024 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "f4ff9ca441b14a468709c0efd0ae984e" old_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: NON_VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: true } } } new_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } attrs { replace: true } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } } }
I20251023 20:16:57.307804 20001 raft_consensus.cc:2917] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Aborting config change with OpId 2.34: Aborted: Op aborted
I20251023 20:16:57.307893 20001 raft_consensus.cc:2889] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20251023 20:16:57.307961 20001 raft_consensus.cc:2272] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:16:57.309271 20001 ts_tablet_manager.cc:1916] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:16:57.313036 20001 ts_tablet_manager.cc:1929] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.34
I20251023 20:16:57.313133 20001 log.cc:1199] T f4ff9ca441b14a468709c0efd0ae984e P 116b80452edd43bc9d822d31be0c5c38: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/f4ff9ca441b14a468709c0efd0ae984e
I20251023 20:16:57.313623 18941 catalog_manager.cc:4985] TS 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): tablet f4ff9ca441b14a468709c0efd0ae984e (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:16:58.159348 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:16:58.161887 19202 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "cea0984de40845da94e6f23513009f02"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:53226
I20251023 20:16:58.162081 19202 raft_consensus.cc:606] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:16:58.162516 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:16:58.162724 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:16:58.309727 19056 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:16:58.311889 19128 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
W20251023 20:16:58.312140 19020 consensus_peers.cc:597] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 -> Peer 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839): Couldn't send request to peer 4cccadfc20fb475c9e6c1ce6cd3ba850. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4c390009f384f9f9a3f424a69b2e5ce. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:58.312382 19997 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:58.312496 19202 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251023 20:16:58.312875 19997 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251023 20:16:58.314531 20021 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:58.315152 19128 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:58.315367 19202 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:58.319056 18951 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 reported cstate change: config changed from index -1 to 28, NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } } }
I20251023 20:16:58.320781 19599 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 31, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:16:58.323172 19056 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
W20251023 20:16:58.323210 19565 consensus_peers.cc:597] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 -> Peer 3c36454265a141c0adf91706fa3115e9 (127.15.81.197:33571): Couldn't send request to peer 3c36454265a141c0adf91706fa3115e9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38153cd2d04e4c56b22f87e07a3c2d11. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:16:58.323689 20006 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:58.329684 20027 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 LEADER]: Committing config change with OpId 3.33: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:58.329962 19128 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 3 index: 33. (index mismatch)
I20251023 20:16:58.330411 20029 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:16:58.330299 19056 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:58.333710 19128 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Committing config change with OpId 3.33: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } } }
I20251023 20:16:58.337936 18946 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 31 to 33, NON_VOTER 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) added. New cstate: current_term: 3 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:58.572114 19989 raft_consensus.cc:993] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:58.572301 19989 raft_consensus.cc:1081] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:16:58.572782 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36232
I20251023 20:16:58.572988 19128 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:16:58.573079 19128 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:16:58.575479 19128 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:58.576166 19128 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 5 election: Requested vote from peers cea0984de40845da94e6f23513009f02 (127.15.81.195:37799), 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:16:58.576551 19202 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "cea0984de40845da94e6f23513009f02"
I20251023 20:16:58.576521 19598 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "329b90f727b745ddacc774c89c3a4428"
I20251023 20:16:58.576691 19202 raft_consensus.cc:3055] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Stepping down as leader of term 4
I20251023 20:16:58.576720 19598 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:16:58.576756 19202 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:16:58.576989 19202 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:58.577288 19202 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:16:58.579128 19598 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 5.
I20251023 20:16:58.579495 19202 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 5.
I20251023 20:16:58.579520 19096 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329b90f727b745ddacc774c89c3a4428, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:16:58.579728 19961 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Leader election won for term 5
I20251023 20:16:58.580184 19961 raft_consensus.cc:697] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:16:58.580400 19961 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:16:58.582798 18952 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 4 to 5, leader changed from cea0984de40845da94e6f23513009f02 (127.15.81.195) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 5 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:16:58.829877 20036 ts_tablet_manager.cc:933] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: Initiating tablet copy from peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987)
I20251023 20:16:58.830408 20036 tablet_copy_client.cc:323] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:36987
I20251023 20:16:58.830940 19067 tablet_copy_service.cc:140] P 116b80452edd43bc9d822d31be0c5c38: Received BeginTabletCopySession request for tablet b4c390009f384f9f9a3f424a69b2e5ce from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 ({username='slave'} at 127.0.0.1:55958)
I20251023 20:16:58.831068 19067 tablet_copy_service.cc:161] P 116b80452edd43bc9d822d31be0c5c38: Beginning new tablet copy session on tablet b4c390009f384f9f9a3f424a69b2e5ce from peer 4cccadfc20fb475c9e6c1ce6cd3ba850 at {username='slave'} at 127.0.0.1:55958: session id = 4cccadfc20fb475c9e6c1ce6cd3ba850-b4c390009f384f9f9a3f424a69b2e5ce
I20251023 20:16:58.833185 19067 tablet_copy_source_session.cc:215] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:58.834237 20036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c390009f384f9f9a3f424a69b2e5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:58.839334 20036 tablet_copy_client.cc:806] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:58.839542 20036 tablet_copy_client.cc:670] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:58.840890 20037 tablet_copy_client.cc:1025] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:58.893769 20040 ts_tablet_manager.cc:933] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: Initiating tablet copy from peer 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:16:58.894259 20040 tablet_copy_client.cc:323] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.198:43569
I20251023 20:16:58.894767 19609 tablet_copy_service.cc:140] P 329b90f727b745ddacc774c89c3a4428: Received BeginTabletCopySession request for tablet 38153cd2d04e4c56b22f87e07a3c2d11 from peer 3c36454265a141c0adf91706fa3115e9 ({username='slave'} at 127.0.0.1:40606)
I20251023 20:16:58.894882 19609 tablet_copy_service.cc:161] P 329b90f727b745ddacc774c89c3a4428: Beginning new tablet copy session on tablet 38153cd2d04e4c56b22f87e07a3c2d11 from peer 3c36454265a141c0adf91706fa3115e9 at {username='slave'} at 127.0.0.1:40606: session id = 3c36454265a141c0adf91706fa3115e9-38153cd2d04e4c56b22f87e07a3c2d11
I20251023 20:16:58.896550 19609 tablet_copy_source_session.cc:215] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:16:58.897537 20040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38153cd2d04e4c56b22f87e07a3c2d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:16:58.901365 20040 tablet_copy_client.cc:806] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 0 data blocks...
I20251023 20:16:58.901543 20040 tablet_copy_client.cc:670] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Starting download of 1 WAL segments...
I20251023 20:16:58.902751 20041 tablet_copy_client.cc:1025] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Injecting latency into file download: 2000
I20251023 20:16:58.983788 19599 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 5 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 20:16:58.984263 19961 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:16:58.988098 19202 raft_consensus.cc:1275] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 5 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 20:16:58.988545 19961 consensus_queue.cc:1048] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:00.163034 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:00.166025 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:17:00.166190 19128 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:17:00.166605 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:00.166810 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:00.229931 20052 raft_consensus.cc:993] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:00.230104 20052 raft_consensus.cc:1081] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:00.230618 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:17:00.230793 19056 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:00.230881 19056 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:17:00.233134 19056 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:00.233702 19056 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 5 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:17:00.234063 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:17:00.234213 19128 raft_consensus.cc:3055] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Stepping down as leader of term 4
I20251023 20:17:00.234282 19128 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:00.234488 19128 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:00.234725 19128 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:17:00.236984 19128 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 5.
I20251023 20:17:00.236992 19599 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: true dest_uuid: "329b90f727b745ddacc774c89c3a4428"
I20251023 20:17:00.237234 19599 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:17:00.237358 19020 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:17:00.237644 20048 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 FOLLOWER]: Leader election won for term 5
I20251023 20:17:00.237771 20048 raft_consensus.cc:697] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:00.237957 20048 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:00.239533 19599 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 5.
I20251023 20:17:00.240439 18952 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 4 to 5, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 5 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:00.654119 19128 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 20:17:00.654598 20048 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:00.658365 19599 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 5 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251023 20:17:00.659343 20048 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:00.841220 20036 tablet_copy_client.cc:538] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:00.843493 20036 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap starting.
I20251023 20:17:00.871634 20036 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=117 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:00.871944 20036 tablet_bootstrap.cc:492] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: Bootstrap complete.
I20251023 20:17:00.872159 20036 ts_tablet_manager.cc:1403] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent bootstrapping tablet: real 0.029s user 0.023s sys 0.004s
I20251023 20:17:00.872654 20036 raft_consensus.cc:359] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:17:00.872797 20036 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4cccadfc20fb475c9e6c1ce6cd3ba850, State: Initialized, Role: LEARNER
I20251023 20:17:00.872982 20036 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: true } }
I20251023 20:17:00.873392 20036 ts_tablet_manager.cc:1434] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20251023 20:17:00.873955 19067 tablet_copy_service.cc:342] P 116b80452edd43bc9d822d31be0c5c38: Request end of tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-b4c390009f384f9f9a3f424a69b2e5ce received from {username='slave'} at 127.0.0.1:55958
I20251023 20:17:00.874055 19067 tablet_copy_service.cc:434] P 116b80452edd43bc9d822d31be0c5c38: ending tablet copy session 4cccadfc20fb475c9e6c1ce6cd3ba850-b4c390009f384f9f9a3f424a69b2e5ce on tablet b4c390009f384f9f9a3f424a69b2e5ce with peer 4cccadfc20fb475c9e6c1ce6cd3ba850
I20251023 20:17:00.903050 20040 tablet_copy_client.cc:538] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:00.905745 20040 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: Bootstrap starting.
I20251023 20:17:00.938176 20040 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:00.938431 20040 tablet_bootstrap.cc:492] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: Bootstrap complete.
I20251023 20:17:00.938657 20040 ts_tablet_manager.cc:1403] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: Time spent bootstrapping tablet: real 0.033s user 0.032s sys 0.000s
I20251023 20:17:00.939189 20040 raft_consensus.cc:359] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:17:00.939325 20040 raft_consensus.cc:740] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 3c36454265a141c0adf91706fa3115e9, State: Initialized, Role: LEARNER
I20251023 20:17:00.939479 20040 consensus_queue.cc:260] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: NON_VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: true } }
I20251023 20:17:00.939913 20040 ts_tablet_manager.cc:1434] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:00.940464 19609 tablet_copy_service.cc:342] P 329b90f727b745ddacc774c89c3a4428: Request end of tablet copy session 3c36454265a141c0adf91706fa3115e9-38153cd2d04e4c56b22f87e07a3c2d11 received from {username='slave'} at 127.0.0.1:40606
I20251023 20:17:00.940570 19609 tablet_copy_service.cc:434] P 329b90f727b745ddacc774c89c3a4428: ending tablet copy session 3c36454265a141c0adf91706fa3115e9-38153cd2d04e4c56b22f87e07a3c2d11 on tablet 38153cd2d04e4c56b22f87e07a3c2d11 with peer 3c36454265a141c0adf91706fa3115e9
I20251023 20:17:00.992336 19524 raft_consensus.cc:1217] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.32->[3.33-3.33] Dedup: 3.33->[]
I20251023 20:17:01.308771 19448 raft_consensus.cc:1217] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.28] Dedup: 1.28->[]
I20251023 20:17:01.540661 20069 raft_consensus.cc:1064] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: attempting to promote NON_VOTER 3c36454265a141c0adf91706fa3115e9 to VOTER
I20251023 20:17:01.541646 20069 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 31, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:17:01.543499 19524 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 LEARNER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251023 20:17:01.543576 19054 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251023 20:17:01.543903 19128 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251023 20:17:01.543988 20070 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:17:01.544478 20070 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:17:01.545063 20070 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:17:01.554021 20072 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 LEADER]: Committing config change with OpId 3.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.554558 19056 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Committing config change with OpId 3.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.558617 19524 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Committing config change with OpId 3.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.559487 18947 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:17:01.564949 19128 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Committing config change with OpId 3.34: config changed from index 33 to 34, 3c36454265a141c0adf91706fa3115e9 (127.15.81.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } attrs { replace: true } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.570849 19599 consensus_queue.cc:237] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 31, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } }
I20251023 20:17:01.572482 19524 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20251023 20:17:01.572955 20072 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:01.573875 19054 raft_consensus.cc:1275] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Refusing update from remote peer 329b90f727b745ddacc774c89c3a4428: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
I20251023 20:17:01.574682 20072 consensus_queue.cc:1048] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:01.575080 20069 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 LEADER]: Committing config change with OpId 3.35: config changed from index 34 to 35, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.575618 19524 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Committing config change with OpId 3.35: config changed from index 34 to 35, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.578220 18998 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 38153cd2d04e4c56b22f87e07a3c2d11 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:01.579305 18952 catalog_manager.cc:5649] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 reported cstate change: config changed from index 34 to 35, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New cstate: current_term: 3 leader_uuid: "329b90f727b745ddacc774c89c3a4428" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:17:01.580152 19054 raft_consensus.cc:2955] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Committing config change with OpId 3.35: config changed from index 34 to 35, VOTER f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } } peers { permanent_uuid: "3c36454265a141c0adf91706fa3115e9" member_type: VOTER last_known_addr { host: "127.15.81.197" port: 33571 } attrs { promote: false } } }
I20251023 20:17:01.585743 19113 tablet_service.cc:1552] Processing DeleteTablet for tablet 38153cd2d04e4c56b22f87e07a3c2d11 with delete_type TABLET_DATA_TOMBSTONED (TS f909677d13e94318ad9a3b1b2db05723 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:36224
I20251023 20:17:01.586364 20081 tablet_replica.cc:333] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:01.586526 20081 raft_consensus.cc:2243] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:01.586824 20081 raft_consensus.cc:2272] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:01.587533 20081 ts_tablet_manager.cc:1916] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:17:01.591624 20081 ts_tablet_manager.cc:1929] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.34
I20251023 20:17:01.591714 20081 log.cc:1199] T 38153cd2d04e4c56b22f87e07a3c2d11 P f909677d13e94318ad9a3b1b2db05723: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/38153cd2d04e4c56b22f87e07a3c2d11
I20251023 20:17:01.592343 18939 catalog_manager.cc:4985] TS f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259): tablet 38153cd2d04e4c56b22f87e07a3c2d11 (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:17:01.808919 20082 raft_consensus.cc:1064] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: attempting to promote NON_VOTER 4cccadfc20fb475c9e6c1ce6cd3ba850 to VOTER
I20251023 20:17:01.809667 20082 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:01.811499 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 LEARNER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:17:01.811785 19128 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:17:01.811789 19202 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251023 20:17:01.812296 20068 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:17:01.812529 20083 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:17:01.812796 20082 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251023 20:17:01.815542 20068 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.816445 19448 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.818921 19128 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.819532 19202 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.821465 18951 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 reported cstate change: config changed from index 28 to 29, 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.829730 19054 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:01.831032 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:17:01.831032 19128 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251023 20:17:01.831452 20088 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:17:01.831645 20083 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251023 20:17:01.833680 20089 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.834322 19448 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.834558 19128 raft_consensus.cc:2955] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.836932 18941 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet b4c390009f384f9f9a3f424a69b2e5ce with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:01.838271 18946 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 reported cstate change: config changed from index 29 to 30, VOTER cea0984de40845da94e6f23513009f02 (127.15.81.195) evicted. New cstate: current_term: 1 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } } }
I20251023 20:17:01.840806 19196 tablet_service.cc:1552] Processing DeleteTablet for tablet b4c390009f384f9f9a3f424a69b2e5ce with delete_type TABLET_DATA_TOMBSTONED (TS cea0984de40845da94e6f23513009f02 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:53196
I20251023 20:17:01.845140 20090 tablet_replica.cc:333] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:01.845294 20090 raft_consensus.cc:2243] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:01.845463 20090 raft_consensus.cc:2272] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:01.846019 20090 ts_tablet_manager.cc:1916] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:17:01.849752 20090 ts_tablet_manager.cc:1929] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.29
I20251023 20:17:01.849843 20090 log.cc:1199] T b4c390009f384f9f9a3f424a69b2e5ce P cea0984de40845da94e6f23513009f02: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/b4c390009f384f9f9a3f424a69b2e5ce
I20251023 20:17:01.850410 18938 catalog_manager.cc:4985] TS cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): tablet b4c390009f384f9f9a3f424a69b2e5ce (table test-workload [id=f7183b919b694c279d001a5e3265efa4]) successfully deleted
I20251023 20:17:02.167157 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:02.169624 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:17:02.169787 19056 raft_consensus.cc:606] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:17:02.170212 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:02.170446 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:02.397912 20082 raft_consensus.cc:993] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:02.398051 20082 raft_consensus.cc:1081] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:02.398533 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:17:02.398675 19128 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:02.398758 19128 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:02.401510 19128 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:02.402149 19128 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 2 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:17:02.402490 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:17:02.402680 19056 raft_consensus.cc:3055] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:17:02.402750 19056 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:02.402983 19056 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:02.403314 19056 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:02.405541 19056 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 2.
I20251023 20:17:02.405911 19098 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:17:02.406155 20091 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:17:02.406258 20091 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:02.406462 20091 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:02.408126 19448 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
I20251023 20:17:02.408249 19448 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:02.409346 18947 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 1 to 2, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:02.409715 19448 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 2.
I20251023 20:17:02.807868 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 20:17:02.808375 20091 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:17:02.811455 19056 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251023 20:17:02.811955 20091 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251023 20:17:04.170843 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:04.173238 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:17:04.173396 19128 raft_consensus.cc:606] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:17:04.173837 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:04.174024 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:04.232834 20102 raft_consensus.cc:993] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:04.233011 20101 raft_consensus.cc:1081] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:04.233520 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:17:04.233670 19056 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:04.233773 19056 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:17:04.235634 19056 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:04.236112 19056 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 3 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:17:04.236450 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:17:04.236531 19448 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: true dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
I20251023 20:17:04.236604 19128 raft_consensus.cc:3055] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:17:04.236629 19448 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:17:04.236688 19128 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:04.236956 19128 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:04.237262 19128 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:17:04.239138 19448 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 3.
I20251023 20:17:04.239511 19128 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 3.
I20251023 20:17:04.239511 19020 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, 4cccadfc20fb475c9e6c1ce6cd3ba850; no voters:
I20251023 20:17:04.239766 20082 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:17:04.240130 20082 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:04.240378 20082 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:04.242340 18947 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 2 to 3, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 3 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:04.633409 19128 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 20:17:04.633831 20082 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:17:04.637475 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 3 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251023 20:17:04.637969 20082 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251023 20:17:06.174454 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:06.176710 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:17:06.176898 19056 raft_consensus.cc:606] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:17:06.177353 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:06.177547 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:06.237246 20103 raft_consensus.cc:993] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:06.237396 20103 raft_consensus.cc:1081] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:06.237891 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:17:06.238034 19128 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:06.238097 19128 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:17:06.239775 19128 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:06.240219 19128 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 4 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:17:06.240589 19448 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
I20251023 20:17:06.240561 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:17:06.240741 19448 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:17:06.240767 19056 raft_consensus.cc:3055] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 20:17:06.240818 19056 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:06.241051 19056 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:06.241430 19056 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Advancing to term 4
I20251023 20:17:06.243468 19448 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 4.
I20251023 20:17:06.243862 19056 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 4.
I20251023 20:17:06.243885 19096 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cccadfc20fb475c9e6c1ce6cd3ba850, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:17:06.244125 20101 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 20:17:06.244457 20101 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:06.244649 20101 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:06.246642 18952 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 3 to 4, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 4 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:06.629266 19056 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:17:06.629715 20101 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:17:06.633730 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251023 20:17:06.634248 20115 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251023 20:17:08.177858 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:08.180335 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:17:08.180533 19128 raft_consensus.cc:606] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:17:08.180987 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:08.181183 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:08.521009 20129 raft_consensus.cc:993] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:08.521210 20129 raft_consensus.cc:1081] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:08.521726 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:17:08.521903 19056 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:08.522022 19056 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:17:08.523813 19056 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:08.524315 19056 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 5 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:17:08.524714 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:17:08.524777 19448 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
I20251023 20:17:08.524896 19128 raft_consensus.cc:3055] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Stepping down as leader of term 4
I20251023 20:17:08.525024 19128 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:08.525029 19448 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:17:08.525290 19128 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:08.525585 19128 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Advancing to term 5
I20251023 20:17:08.526933 19448 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 5.
I20251023 20:17:08.527240 19128 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 5.
I20251023 20:17:08.527328 19020 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, 4cccadfc20fb475c9e6c1ce6cd3ba850; no voters:
I20251023 20:17:08.527626 20103 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 FOLLOWER]: Leader election won for term 5
I20251023 20:17:08.527961 20103 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:08.528203 20103 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:08.530354 18947 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 4 to 5, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 5 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:09.056066 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 5 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251023 20:17:09.056500 20103 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:17:09.059918 19128 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251023 20:17:09.060448 20141 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251023 20:17:10.181530 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:10.184129 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:17:10.184320 19056 raft_consensus.cc:606] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:17:10.184777 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:10.185004 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:10.670794 20145 raft_consensus.cc:993] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:10.670933 20145 raft_consensus.cc:1081] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:10.671466 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:17:10.671630 19128 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:10.671741 19128 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:10.673449 19128 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:10.673938 19128 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 6 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:17:10.674255 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 6 candidate_status { last_received { term: 5 index: 34 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:17:10.674345 19448 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 6 candidate_status { last_received { term: 5 index: 34 } } ignore_live_leader: true dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
I20251023 20:17:10.674424 19056 raft_consensus.cc:3055] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Stepping down as leader of term 5
I20251023 20:17:10.674499 19056 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:10.674530 19448 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:10.674733 19056 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 5.34, Last appended by leader: 34, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:10.674983 19056 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:10.676126 19448 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 6.
I20251023 20:17:10.676465 19096 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cccadfc20fb475c9e6c1ce6cd3ba850, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:17:10.676681 20129 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Leader election won for term 6
I20251023 20:17:10.676705 19056 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 6.
I20251023 20:17:10.677008 20129 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:10.677240 20129 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 5.34, Last appended by leader: 34, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:10.679961 18947 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 5 to 6, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 6 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:11.107875 19056 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251023 20:17:11.108384 20129 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:11.112116 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 6 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251023 20:17:11.112658 20129 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251023 20:17:12.185351 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:12.187732 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:17:12.187956 19128 raft_consensus.cc:606] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Received request to transfer leadership to TS 116b80452edd43bc9d822d31be0c5c38
I20251023 20:17:12.188402 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:12.188577 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:12.557503 20158 raft_consensus.cc:993] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: : Instructing follower 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:12.557644 20158 raft_consensus.cc:1081] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Signalling peer 116b80452edd43bc9d822d31be0c5c38 to start an election
I20251023 20:17:12.558208 19056 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
from {username='slave'} at 127.0.0.1:55944
I20251023 20:17:12.558386 19056 raft_consensus.cc:493] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:12.558496 19056 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Advancing to term 7
I20251023 20:17:12.560240 19056 raft_consensus.cc:515] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:12.560976 19056 leader_election.cc:290] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 7 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 4cccadfc20fb475c9e6c1ce6cd3ba850 (127.15.81.196:34839)
I20251023 20:17:12.561350 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 7 candidate_status { last_received { term: 6 index: 35 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:17:12.561427 19448 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c390009f384f9f9a3f424a69b2e5ce" candidate_uuid: "116b80452edd43bc9d822d31be0c5c38" candidate_term: 7 candidate_status { last_received { term: 6 index: 35 } } ignore_live_leader: true dest_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850"
I20251023 20:17:12.561538 19128 raft_consensus.cc:3055] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Stepping down as leader of term 6
I20251023 20:17:12.561625 19448 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 6 FOLLOWER]: Advancing to term 7
I20251023 20:17:12.561641 19128 raft_consensus.cc:740] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:12.561892 19128 consensus_queue.cc:260] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 6.35, Last appended by leader: 35, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:12.562162 19128 raft_consensus.cc:3060] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Advancing to term 7
I20251023 20:17:12.564234 19128 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 7.
I20251023 20:17:12.564669 19020 leader_election.cc:304] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 116b80452edd43bc9d822d31be0c5c38, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:17:12.564936 20145 raft_consensus.cc:2804] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 7 FOLLOWER]: Leader election won for term 7
I20251023 20:17:12.565308 20145 raft_consensus.cc:697] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 7 LEADER]: Becoming Leader. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:12.565565 20145 consensus_queue.cc:237] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 6.35, Last appended by leader: 35, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } }
I20251023 20:17:12.567984 18947 catalog_manager.cc:5649] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 reported cstate change: term changed from 6 to 7, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193). New cstate: current_term: 7 leader_uuid: "116b80452edd43bc9d822d31be0c5c38" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:12.581995 19448 raft_consensus.cc:2468] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 116b80452edd43bc9d822d31be0c5c38 in term 7.
I20251023 20:17:12.945632 19448 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 7 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 7 index: 36. (index mismatch)
I20251023 20:17:12.946131 20145 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cccadfc20fb475c9e6c1ce6cd3ba850" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 34839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 20:17:12.949766 19128 raft_consensus.cc:1275] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 7 FOLLOWER]: Refusing update from remote peer 116b80452edd43bc9d822d31be0c5c38: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 7 index: 36. (index mismatch)
I20251023 20:17:12.950305 20145 consensus_queue.cc:1048] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 20:17:14.188944 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:14.191447 19056 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
mode: GRACEFUL
new_leader_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:49642
I20251023 20:17:14.191623 19056 raft_consensus.cc:606] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Received request to transfer leadership to TS f909677d13e94318ad9a3b1b2db05723
I20251023 20:17:14.192052 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:14.192238 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:14.312419 20160 raft_consensus.cc:993] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: : Instructing follower f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:14.312536 20160 raft_consensus.cc:1081] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Signalling peer f909677d13e94318ad9a3b1b2db05723 to start an election
I20251023 20:17:14.313055 19128 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
from {username='slave'} at 127.0.0.1:36246
I20251023 20:17:14.313202 19128 raft_consensus.cc:493] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:14.313266 19128 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:14.315004 19128 raft_consensus.cc:515] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:14.315577 19128 leader_election.cc:290] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 6 election: Requested vote from peers 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987), 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:17:14.315934 19056 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: true dest_uuid: "116b80452edd43bc9d822d31be0c5c38"
I20251023 20:17:14.315963 19598 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7184738166284bf3a71cb7c0d38cd863" candidate_uuid: "f909677d13e94318ad9a3b1b2db05723" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: true dest_uuid: "329b90f727b745ddacc774c89c3a4428"
I20251023 20:17:14.316090 19056 raft_consensus.cc:3055] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Stepping down as leader of term 5
I20251023 20:17:14.316102 19598 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:14.316165 19056 raft_consensus.cc:740] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 116b80452edd43bc9d822d31be0c5c38, State: Running, Role: LEADER
I20251023 20:17:14.316380 19056 consensus_queue.cc:260] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:14.316620 19056 raft_consensus.cc:3060] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:14.317926 19598 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 6.
I20251023 20:17:14.318334 19096 leader_election.cc:304] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329b90f727b745ddacc774c89c3a4428, f909677d13e94318ad9a3b1b2db05723; no voters:
I20251023 20:17:14.318435 19056 raft_consensus.cc:2468] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f909677d13e94318ad9a3b1b2db05723 in term 6.
I20251023 20:17:14.318547 20158 raft_consensus.cc:2804] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Leader election won for term 6
I20251023 20:17:14.318898 20158 raft_consensus.cc:697] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Becoming Leader. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:14.319099 20158 consensus_queue.cc:237] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:14.321201 18952 catalog_manager.cc:5649] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 reported cstate change: term changed from 5 to 6, leader changed from 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193) to f909677d13e94318ad9a3b1b2db05723 (127.15.81.194). New cstate: current_term: 6 leader_uuid: "f909677d13e94318ad9a3b1b2db05723" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:14.776777 19598 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251023 20:17:14.777321 20158 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 20:17:14.781608 19056 raft_consensus.cc:1275] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Refusing update from remote peer f909677d13e94318ad9a3b1b2db05723: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251023 20:17:14.782155 20182 consensus_queue.cc:1048] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116b80452edd43bc9d822d31be0c5c38" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251023 20:17:16.192654 19007 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:16.195228 19128 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
mode: GRACEFUL
new_leader_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:49246
I20251023 20:17:16.195380 19128 raft_consensus.cc:606] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 LEADER]: Received request to transfer leadership to TS cea0984de40845da94e6f23513009f02
I20251023 20:17:16.195812 19007 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:16.196005 19007 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:16.536010 20186 raft_consensus.cc:993] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: : Instructing follower cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:17:16.536139 20186 raft_consensus.cc:1081] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 LEADER]: Signalling peer cea0984de40845da94e6f23513009f02 to start an election
I20251023 20:17:16.536626 19202 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0"
dest_uuid: "cea0984de40845da94e6f23513009f02"
from {username='slave'} at 127.0.0.1:53212
I20251023 20:17:16.536795 19202 raft_consensus.cc:493] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:16.536919 19202 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:16.538802 19202 raft_consensus.cc:515] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:16.539419 19202 leader_election.cc:290] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 6 election: Requested vote from peers f909677d13e94318ad9a3b1b2db05723 (127.15.81.194:33259), 329b90f727b745ddacc774c89c3a4428 (127.15.81.198:43569)
I20251023 20:17:16.539804 19599 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: true dest_uuid: "329b90f727b745ddacc774c89c3a4428"
I20251023 20:17:16.539793 19128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c0d82f5ba4a476583099f65588d84f0" candidate_uuid: "cea0984de40845da94e6f23513009f02" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: true dest_uuid: "f909677d13e94318ad9a3b1b2db05723"
I20251023 20:17:16.540001 19599 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:16.540055 19128 raft_consensus.cc:3055] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 LEADER]: Stepping down as leader of term 5
I20251023 20:17:16.540138 19128 raft_consensus.cc:740] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f909677d13e94318ad9a3b1b2db05723, State: Running, Role: LEADER
I20251023 20:17:16.540350 19128 consensus_queue.cc:260] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:16.540659 19128 raft_consensus.cc:3060] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 5 FOLLOWER]: Advancing to term 6
I20251023 20:17:16.541923 19599 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 6.
I20251023 20:17:16.542289 19170 leader_election.cc:304] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329b90f727b745ddacc774c89c3a4428, cea0984de40845da94e6f23513009f02; no voters:
I20251023 20:17:16.542520 20127 raft_consensus.cc:2804] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 6 FOLLOWER]: Leader election won for term 6
I20251023 20:17:16.542560 19128 raft_consensus.cc:2468] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea0984de40845da94e6f23513009f02 in term 6.
I20251023 20:17:16.542889 20127 raft_consensus.cc:697] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 6 LEADER]: Becoming Leader. State: Replica: cea0984de40845da94e6f23513009f02, State: Running, Role: LEADER
I20251023 20:17:16.543120 20127 consensus_queue.cc:237] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } }
I20251023 20:17:16.545195 18952 catalog_manager.cc:5649] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 reported cstate change: term changed from 5 to 6, leader changed from f909677d13e94318ad9a3b1b2db05723 (127.15.81.194) to cea0984de40845da94e6f23513009f02 (127.15.81.195). New cstate: current_term: 6 leader_uuid: "cea0984de40845da94e6f23513009f02" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "f909677d13e94318ad9a3b1b2db05723" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea0984de40845da94e6f23513009f02" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329b90f727b745ddacc774c89c3a4428" member_type: VOTER last_known_addr { host: "127.15.81.198" port: 43569 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/auto_rebalancer-test.cc:195: Failure
Expected equality of these values:
0
NumMovesScheduled(leader_idx, BalanceThreadType::LEADER_REBALANCE)
Which is: 1
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/auto_rebalancer-test.cc:575: Failure
Expected: CheckNoLeaderMovesScheduled() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20251023 20:17:16.689530 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
W20251023 20:17:16.701388 19565 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20251023 20:17:16.702574 19565 consensus_peers.cc:597] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 -> Peer 116b80452edd43bc9d822d31be0c5c38 (127.15.81.193:36987): Couldn't send request to peer 116b80452edd43bc9d822d31be0c5c38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:16.704391 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:16.704600 15687 tablet_replica.cc:333] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:17:16.704744 15687 raft_consensus.cc:2243] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.704907 15687 raft_consensus.cc:2272] T 7184738166284bf3a71cb7c0d38cd863 P 116b80452edd43bc9d822d31be0c5c38 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.705428 15687 tablet_replica.cc:333] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:17:16.705560 15687 raft_consensus.cc:2243] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.705770 15687 raft_consensus.cc:2272] T 37a00c71a3934405917c05725c94dde5 P 116b80452edd43bc9d822d31be0c5c38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.706194 15687 tablet_replica.cc:333] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:17:16.706307 15687 raft_consensus.cc:2243] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.706519 15687 raft_consensus.cc:2272] T fc60b796f0d946fbb49d15812e88ac60 P 116b80452edd43bc9d822d31be0c5c38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.706911 15687 tablet_replica.cc:333] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:17:16.707023 15687 raft_consensus.cc:2243] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.707137 15687 raft_consensus.cc:2272] T 508756851fc14f93a7d310b11beb15cb P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.707549 15687 tablet_replica.cc:333] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:17:16.707675 15687 raft_consensus.cc:2243] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.707842 15687 raft_consensus.cc:2272] T 38153cd2d04e4c56b22f87e07a3c2d11 P 116b80452edd43bc9d822d31be0c5c38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.708220 15687 tablet_replica.cc:333] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38: stopping tablet replica
I20251023 20:17:16.708359 15687 raft_consensus.cc:2243] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 7 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.708570 15687 raft_consensus.cc:2272] T b4c390009f384f9f9a3f424a69b2e5ce P 116b80452edd43bc9d822d31be0c5c38 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.724182 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:17:16.729349 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:17:16.751839 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:16.752071 15687 tablet_replica.cc:333] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:16.752266 15687 raft_consensus.cc:2243] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 6 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.752519 15687 raft_consensus.cc:2272] T 7184738166284bf3a71cb7c0d38cd863 P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.753082 15687 tablet_replica.cc:333] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:16.753192 15687 raft_consensus.cc:2243] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.753340 15687 raft_consensus.cc:2272] T f4ff9ca441b14a468709c0efd0ae984e P f909677d13e94318ad9a3b1b2db05723 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.753742 15687 tablet_replica.cc:333] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:16.753832 15687 raft_consensus.cc:2243] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 3 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.754012 15687 raft_consensus.cc:2272] T 508756851fc14f93a7d310b11beb15cb P f909677d13e94318ad9a3b1b2db05723 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.754379 15687 tablet_replica.cc:333] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:16.754463 15687 raft_consensus.cc:2243] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.754565 15687 raft_consensus.cc:2272] T 37f44fa5d63049edbd7ad28d358075fb P f909677d13e94318ad9a3b1b2db05723 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.754901 15687 tablet_replica.cc:333] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:16.754982 15687 raft_consensus.cc:2243] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 7 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.755066 15687 raft_consensus.cc:2272] T b4c390009f384f9f9a3f424a69b2e5ce P f909677d13e94318ad9a3b1b2db05723 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.755395 15687 tablet_replica.cc:333] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723: stopping tablet replica
I20251023 20:17:16.755493 15687 raft_consensus.cc:2243] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.755576 15687 raft_consensus.cc:2272] T 7c0d82f5ba4a476583099f65588d84f0 P f909677d13e94318ad9a3b1b2db05723 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.770884 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:17:16.775517 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
W20251023 20:17:16.778560 19487 consensus_peers.cc:597] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 -> Peer cea0984de40845da94e6f23513009f02 (127.15.81.195:37799): Couldn't send request to peer cea0984de40845da94e6f23513009f02. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:16.785307 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:16.785880 15687 tablet_replica.cc:333] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:16.786020 15687 raft_consensus.cc:2243] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 6 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.786211 15687 pending_rounds.cc:70] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Trying to abort 1 pending ops.
I20251023 20:17:16.786248 15687 pending_rounds.cc:77] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02: Aborting op as it isn't in flight: id { term: 6 index: 36 } timestamp: 7214082607282548736 op_type: NO_OP noop_request { }
I20251023 20:17:16.786367 15687 raft_consensus.cc:2889] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 6 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:16.786448 15687 raft_consensus.cc:2272] T 7c0d82f5ba4a476583099f65588d84f0 P cea0984de40845da94e6f23513009f02 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.786972 15687 tablet_replica.cc:333] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:16.787107 15687 raft_consensus.cc:2243] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.787263 15687 raft_consensus.cc:2272] T fc60b796f0d946fbb49d15812e88ac60 P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.787663 15687 tablet_replica.cc:333] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:16.787783 15687 raft_consensus.cc:2243] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.787998 15687 raft_consensus.cc:2272] T 37f44fa5d63049edbd7ad28d358075fb P cea0984de40845da94e6f23513009f02 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.788409 15687 tablet_replica.cc:333] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:16.788528 15687 raft_consensus.cc:2243] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.788730 15687 raft_consensus.cc:2272] T 887ca8402e81417c94da83fdc1b75862 P cea0984de40845da94e6f23513009f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.789158 15687 tablet_replica.cc:333] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:16.789276 15687 raft_consensus.cc:2243] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.789402 15687 raft_consensus.cc:2272] T 461a7ab7c21c40fbb4ef24809dad5831 P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.789767 15687 tablet_replica.cc:333] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02: stopping tablet replica
I20251023 20:17:16.789875 15687 raft_consensus.cc:2243] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.790040 15687 raft_consensus.cc:2272] T 2a9aada4eefa459ea9801d5506a2b82e P cea0984de40845da94e6f23513009f02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.820358 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:17:16.825268 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:17:16.833253 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:16.833470 15687 tablet_replica.cc:333] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850: stopping tablet replica
I20251023 20:17:16.833631 15687 raft_consensus.cc:2243] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.833863 15687 raft_consensus.cc:2272] T fc60b796f0d946fbb49d15812e88ac60 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.834388 15687 tablet_replica.cc:333] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850: stopping tablet replica
I20251023 20:17:16.834513 15687 raft_consensus.cc:2243] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.834664 15687 raft_consensus.cc:2272] T 2a9aada4eefa459ea9801d5506a2b82e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.835047 15687 tablet_replica.cc:333] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850: stopping tablet replica
I20251023 20:17:16.835180 15687 raft_consensus.cc:2243] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 7 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.835321 15687 raft_consensus.cc:2272] T b4c390009f384f9f9a3f424a69b2e5ce P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.835688 15687 tablet_replica.cc:333] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850: stopping tablet replica
I20251023 20:17:16.835803 15687 raft_consensus.cc:2243] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.836019 15687 raft_consensus.cc:2272] T f4ff9ca441b14a468709c0efd0ae984e P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.836392 15687 tablet_replica.cc:333] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850: stopping tablet replica
I20251023 20:17:16.836503 15687 raft_consensus.cc:2243] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.836644 15687 raft_consensus.cc:2272] T 887ca8402e81417c94da83fdc1b75862 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.837041 15687 tablet_replica.cc:333] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850: stopping tablet replica
I20251023 20:17:16.837152 15687 raft_consensus.cc:2243] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.837344 15687 raft_consensus.cc:2272] T 461a7ab7c21c40fbb4ef24809dad5831 P 4cccadfc20fb475c9e6c1ce6cd3ba850 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.851589 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:17:16.854877 15687 tablet_server.cc:178] TabletServer@127.15.81.197:0 shutting down...
I20251023 20:17:16.862880 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:16.863082 15687 tablet_replica.cc:333] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9: stopping tablet replica
I20251023 20:17:16.863231 15687 raft_consensus.cc:2243] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.863418 15687 raft_consensus.cc:2272] T 38153cd2d04e4c56b22f87e07a3c2d11 P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.863858 15687 tablet_replica.cc:333] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9: stopping tablet replica
I20251023 20:17:16.864004 15687 raft_consensus.cc:2243] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.864182 15687 raft_consensus.cc:2272] T 37f44fa5d63049edbd7ad28d358075fb P 3c36454265a141c0adf91706fa3115e9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.864575 15687 tablet_replica.cc:333] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9: stopping tablet replica
I20251023 20:17:16.864691 15687 raft_consensus.cc:2243] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.864879 15687 raft_consensus.cc:2272] T 37a00c71a3934405917c05725c94dde5 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.865265 15687 tablet_replica.cc:333] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9: stopping tablet replica
I20251023 20:17:16.865394 15687 raft_consensus.cc:2243] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.865542 15687 raft_consensus.cc:2272] T 461a7ab7c21c40fbb4ef24809dad5831 P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.865903 15687 tablet_replica.cc:333] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9: stopping tablet replica
I20251023 20:17:16.866014 15687 raft_consensus.cc:2243] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.866227 15687 raft_consensus.cc:2272] T 2a9aada4eefa459ea9801d5506a2b82e P 3c36454265a141c0adf91706fa3115e9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.866604 15687 tablet_replica.cc:333] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9: stopping tablet replica
I20251023 20:17:16.866714 15687 raft_consensus.cc:2243] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.866859 15687 raft_consensus.cc:2272] T 508756851fc14f93a7d310b11beb15cb P 3c36454265a141c0adf91706fa3115e9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.881021 15687 tablet_server.cc:195] TabletServer@127.15.81.197:0 shutdown complete.
I20251023 20:17:16.885111 15687 tablet_server.cc:178] TabletServer@127.15.81.198:0 shutting down...
I20251023 20:17:16.909142 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:16.909441 15687 tablet_replica.cc:333] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428: stopping tablet replica
I20251023 20:17:16.909631 15687 raft_consensus.cc:2243] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.909844 15687 raft_consensus.cc:2272] T 7c0d82f5ba4a476583099f65588d84f0 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.910394 15687 tablet_replica.cc:333] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428: stopping tablet replica
I20251023 20:17:16.910508 15687 raft_consensus.cc:2243] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.910650 15687 raft_consensus.cc:2272] T 7184738166284bf3a71cb7c0d38cd863 P 329b90f727b745ddacc774c89c3a4428 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.911194 15687 tablet_replica.cc:333] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428: stopping tablet replica
I20251023 20:17:16.911334 15687 raft_consensus.cc:2243] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.911500 15687 raft_consensus.cc:2272] T f4ff9ca441b14a468709c0efd0ae984e P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.911911 15687 tablet_replica.cc:333] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428: stopping tablet replica
I20251023 20:17:16.912050 15687 raft_consensus.cc:2243] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.912298 15687 raft_consensus.cc:2272] T 38153cd2d04e4c56b22f87e07a3c2d11 P 329b90f727b745ddacc774c89c3a4428 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.912685 15687 tablet_replica.cc:333] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428: stopping tablet replica
I20251023 20:17:16.912825 15687 raft_consensus.cc:2243] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.913053 15687 raft_consensus.cc:2272] T 37a00c71a3934405917c05725c94dde5 P 329b90f727b745ddacc774c89c3a4428 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.913450 15687 tablet_replica.cc:333] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428: stopping tablet replica
I20251023 20:17:16.913550 15687 raft_consensus.cc:2243] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:16.913702 15687 raft_consensus.cc:2272] T 887ca8402e81417c94da83fdc1b75862 P 329b90f727b745ddacc774c89c3a4428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.927397 15687 tablet_server.cc:195] TabletServer@127.15.81.198:0 shutdown complete.
I20251023 20:17:16.930720 15687 master.cc:561] Master@127.15.81.254:45023 shutting down...
I20251023 20:17:16.937937 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:16.938087 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:16.938151 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P e3bc3766d3674c358cb246e89a6996df: stopping tablet replica
I20251023 20:17:16.941408 15687 master.cc:583] Master@127.15.81.254:45023 shutdown complete.
I20251023 20:17:16.947247 15687 test_util.cc:183] -----------------------------------------------
I20251023 20:17:16.947294 15687 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761250506895969-15687-0
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer (37164 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingUnstableCluster
I20251023 20:17:16.949004 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:45449
I20251023 20:17:16.949440 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:16.951678 20196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:16.951826 20197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:16.951853 20199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:16.952034 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:16.952275 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:16.952328 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:16.952351 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250636952349 us; error 0 us; skew 500 ppm
I20251023 20:17:16.952446 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:16.953416 15687 webserver.cc:492] Webserver started at http://127.15.81.254:43705/ using document root <none> and password file <none>
I20251023 20:17:16.953581 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:16.953629 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:16.953702 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:16.954185 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "b45efb26b0f9484a87eec770f30d8500"
format_stamp: "Formatted at 2025-10-23 20:17:16 on dist-test-slave-jr94"
I20251023 20:17:16.956198 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:16.957446 20204 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:16.957728 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:16.957808 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "b45efb26b0f9484a87eec770f30d8500"
format_stamp: "Formatted at 2025-10-23 20:17:16 on dist-test-slave-jr94"
I20251023 20:17:16.957901 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:16.972779 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:16.973174 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:16.983111 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:45449
I20251023 20:17:16.983122 20255 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:45449 every 8 connection(s)
I20251023 20:17:16.984124 20256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:16.987603 20256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500: Bootstrap starting.
I20251023 20:17:16.988739 20256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:16.990087 20256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500: No bootstrap required, opened a new log
I20251023 20:17:16.990523 20256 raft_consensus.cc:359] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45efb26b0f9484a87eec770f30d8500" member_type: VOTER }
I20251023 20:17:16.990624 20256 raft_consensus.cc:385] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:16.990658 20256 raft_consensus.cc:740] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45efb26b0f9484a87eec770f30d8500, State: Initialized, Role: FOLLOWER
I20251023 20:17:16.990798 20256 consensus_queue.cc:260] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45efb26b0f9484a87eec770f30d8500" member_type: VOTER }
I20251023 20:17:16.990901 20256 raft_consensus.cc:399] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:17:16.990938 20256 raft_consensus.cc:493] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:17:16.990981 20256 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:16.992372 20256 raft_consensus.cc:515] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45efb26b0f9484a87eec770f30d8500" member_type: VOTER }
I20251023 20:17:16.992525 20256 leader_election.cc:304] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b45efb26b0f9484a87eec770f30d8500; no voters:
I20251023 20:17:16.992808 20256 leader_election.cc:290] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:17:16.992928 20259 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:16.993307 20259 raft_consensus.cc:697] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 1 LEADER]: Becoming Leader. State: Replica: b45efb26b0f9484a87eec770f30d8500, State: Running, Role: LEADER
I20251023 20:17:16.993496 20259 consensus_queue.cc:237] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45efb26b0f9484a87eec770f30d8500" member_type: VOTER }
I20251023 20:17:16.993566 20256 sys_catalog.cc:565] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:17:16.994165 20260 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b45efb26b0f9484a87eec770f30d8500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45efb26b0f9484a87eec770f30d8500" member_type: VOTER } }
I20251023 20:17:16.994244 20261 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b45efb26b0f9484a87eec770f30d8500. Latest consensus state: current_term: 1 leader_uuid: "b45efb26b0f9484a87eec770f30d8500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45efb26b0f9484a87eec770f30d8500" member_type: VOTER } }
I20251023 20:17:16.994414 20260 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:16.994426 20261 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:16.997426 20264 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:17:16.998404 20264 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:17:17.000016 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:17:17.000936 20264 catalog_manager.cc:1357] Generated new cluster ID: db7b64a427fc43299cc7d989b0495093
I20251023 20:17:17.000994 20264 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:17:17.013100 20264 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:17:17.013664 20264 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:17:17.020030 20264 catalog_manager.cc:6022] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500: Generated new TSK 0
I20251023 20:17:17.020188 20264 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:17:17.032558 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:17.034917 20277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:17.035346 20278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:17.035636 15687 server_base.cc:1047] running on GCE node
W20251023 20:17:17.035610 20280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:17.038405 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:17.038465 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:17.038487 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250637038485 us; error 0 us; skew 500 ppm
I20251023 20:17:17.038615 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:17.039629 15687 webserver.cc:492] Webserver started at http://127.15.81.193:33429/ using document root <none> and password file <none>
I20251023 20:17:17.039819 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:17.039872 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:17.039985 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:17.040534 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "6d123552770c4dccbf215da899937dee"
format_stamp: "Formatted at 2025-10-23 20:17:17 on dist-test-slave-jr94"
I20251023 20:17:17.042862 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:17.044184 20285 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.044492 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:17.044579 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "6d123552770c4dccbf215da899937dee"
format_stamp: "Formatted at 2025-10-23 20:17:17 on dist-test-slave-jr94"
I20251023 20:17:17.044668 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:17.064096 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:17.064481 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:17.064950 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:17.065682 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:17.065734 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.065800 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:17.065834 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.075920 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:32959
I20251023 20:17:17.075939 20347 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:32959 every 8 connection(s)
I20251023 20:17:17.077224 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:17.079921 20352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:17.079993 20353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:17.081017 20355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:17.081291 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:17.081557 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:17.081622 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:17.081651 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250637081648 us; error 0 us; skew 500 ppm
I20251023 20:17:17.081786 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:17.083035 15687 webserver.cc:492] Webserver started at http://127.15.81.194:36931/ using document root <none> and password file <none>
I20251023 20:17:17.083180 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:17.083223 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:17.083292 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:17.083758 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "088a0d55fd23477b81facff331c2a822"
format_stamp: "Formatted at 2025-10-23 20:17:17 on dist-test-slave-jr94"
I20251023 20:17:17.085748 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 20:17:17.086961 20360 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.089843 20348 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45449
I20251023 20:17:17.090016 20348 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:17.090639 20348 heartbeater.cc:507] Master 127.15.81.254:45449 requested a full tablet report, sending...
I20251023 20:17:17.091593 20221 ts_manager.cc:194] Registered new tserver with Master: 6d123552770c4dccbf215da899937dee (127.15.81.193:32959)
I20251023 20:17:17.092455 20221 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46070
I20251023 20:17:17.106395 15687 fs_manager.cc:730] Time spent opening block manager: real 0.020s user 0.016s sys 0.003s
I20251023 20:17:17.106474 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "088a0d55fd23477b81facff331c2a822"
format_stamp: "Formatted at 2025-10-23 20:17:17 on dist-test-slave-jr94"
I20251023 20:17:17.106571 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:17.119505 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:17.119879 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:17.120272 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:17.120914 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:17.120962 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.121022 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:17.121057 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.131230 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:43973
I20251023 20:17:17.131259 20422 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:43973 every 8 connection(s)
I20251023 20:17:17.132477 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:17.134687 20426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:17.134874 20427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:17.135727 20429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:17.136041 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:17.136302 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:17.136372 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:17.136404 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250637136401 us; error 0 us; skew 500 ppm
I20251023 20:17:17.136497 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:17.137562 15687 webserver.cc:492] Webserver started at http://127.15.81.195:39379/ using document root <none> and password file <none>
I20251023 20:17:17.137725 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:17.137799 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:17.137915 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:17.138007 20423 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45449
I20251023 20:17:17.138109 20423 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:17.138437 20423 heartbeater.cc:507] Master 127.15.81.254:45449 requested a full tablet report, sending...
I20251023 20:17:17.138618 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "d5d5f54dc7d945c7b306d794eaa411bf"
format_stamp: "Formatted at 2025-10-23 20:17:17 on dist-test-slave-jr94"
I20251023 20:17:17.139195 20221 ts_manager.cc:194] Registered new tserver with Master: 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
I20251023 20:17:17.139879 20221 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46084
I20251023 20:17:17.141201 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:17:17.142539 20434 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.142800 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:17.142885 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "d5d5f54dc7d945c7b306d794eaa411bf"
format_stamp: "Formatted at 2025-10-23 20:17:17 on dist-test-slave-jr94"
I20251023 20:17:17.142969 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:17.152364 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:17.152719 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:17.153242 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:17.153940 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:17.153992 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.154057 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:17.154106 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:17.165458 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:34855
I20251023 20:17:17.165480 20496 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:34855 every 8 connection(s)
I20251023 20:17:17.171177 20497 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45449
I20251023 20:17:17.171268 20497 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:17.171514 20497 heartbeater.cc:507] Master 127.15.81.254:45449 requested a full tablet report, sending...
I20251023 20:17:17.172134 20221 ts_manager.cc:194] Registered new tserver with Master: d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855)
I20251023 20:17:17.172711 20221 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46090
I20251023 20:17:17.172788 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006646573s
I20251023 20:17:18.093703 20348 heartbeater.cc:499] Master 127.15.81.254:45449 was elected leader, sending a full tablet report...
I20251023 20:17:18.141183 20423 heartbeater.cc:499] Master 127.15.81.254:45449 was elected leader, sending a full tablet report...
I20251023 20:17:18.173702 20497 heartbeater.cc:499] Master 127.15.81.254:45449 was elected leader, sending a full tablet report...
I20251023 20:17:18.199589 15687 test_util.cc:276] Using random seed: 514226962
I20251023 20:17:18.217644 20221 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46104:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:17:18.218261 20221 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:17:18.235977 20462 tablet_service.cc:1505] Processing CreateTablet for tablet 71cf935bdaff41f5a0f284b97b9e69df (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 20:17:18.236141 20460 tablet_service.cc:1505] Processing CreateTablet for tablet 5de6d5c5a8f642379e215d09277b4224 (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 20:17:18.236423 20462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cf935bdaff41f5a0f284b97b9e69df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.236863 20461 tablet_service.cc:1505] Processing CreateTablet for tablet 81d51f1bd2494650914b71b074d058fb (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 20:17:18.237233 20461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d51f1bd2494650914b71b074d058fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.240983 20460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de6d5c5a8f642379e215d09277b4224. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.245805 20517 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: Bootstrap starting.
I20251023 20:17:18.248747 20517 tablet_bootstrap.cc:654] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.254493 20517 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: No bootstrap required, opened a new log
I20251023 20:17:18.254640 20517 ts_tablet_manager.cc:1403] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20251023 20:17:18.255405 20517 raft_consensus.cc:359] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.255555 20517 raft_consensus.cc:385] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.255620 20517 raft_consensus.cc:740] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d5f54dc7d945c7b306d794eaa411bf, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.255822 20517 consensus_queue.cc:260] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.259900 20313 tablet_service.cc:1505] Processing CreateTablet for tablet 71cf935bdaff41f5a0f284b97b9e69df (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 20:17:18.260006 20312 tablet_service.cc:1505] Processing CreateTablet for tablet 81d51f1bd2494650914b71b074d058fb (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 20:17:18.260314 20313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cf935bdaff41f5a0f284b97b9e69df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.260327 20388 tablet_service.cc:1505] Processing CreateTablet for tablet 71cf935bdaff41f5a0f284b97b9e69df (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251023 20:17:18.260740 20388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cf935bdaff41f5a0f284b97b9e69df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.260658 20311 tablet_service.cc:1505] Processing CreateTablet for tablet 5de6d5c5a8f642379e215d09277b4224 (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 20:17:18.260982 20386 tablet_service.cc:1505] Processing CreateTablet for tablet 5de6d5c5a8f642379e215d09277b4224 (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251023 20:17:18.261405 20386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de6d5c5a8f642379e215d09277b4224. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.266633 20520 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee: Bootstrap starting.
I20251023 20:17:18.267098 20517 ts_tablet_manager.cc:1434] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20251023 20:17:18.260988 20311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de6d5c5a8f642379e215d09277b4224. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.260676 20387 tablet_service.cc:1505] Processing CreateTablet for tablet 81d51f1bd2494650914b71b074d058fb (DEFAULT_TABLE table=test-workload [id=e1d7ddb95f50475b852939e01ea26256]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251023 20:17:18.267757 20387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d51f1bd2494650914b71b074d058fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.272943 20312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d51f1bd2494650914b71b074d058fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:18.275915 20520 tablet_bootstrap.cc:654] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.267355 20517 tablet_bootstrap.cc:492] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf: Bootstrap starting.
I20251023 20:17:18.278798 20517 tablet_bootstrap.cc:654] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.279346 20521 tablet_bootstrap.cc:492] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822: Bootstrap starting.
I20251023 20:17:18.279882 20520 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee: No bootstrap required, opened a new log
I20251023 20:17:18.280062 20520 ts_tablet_manager.cc:1403] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.004s
I20251023 20:17:18.280539 20517 tablet_bootstrap.cc:492] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf: No bootstrap required, opened a new log
I20251023 20:17:18.280655 20517 ts_tablet_manager.cc:1403] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20251023 20:17:18.280965 20521 tablet_bootstrap.cc:654] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.280830 20520 raft_consensus.cc:359] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.281041 20520 raft_consensus.cc:385] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.281109 20520 raft_consensus.cc:740] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d123552770c4dccbf215da899937dee, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.281332 20517 raft_consensus.cc:359] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.281469 20517 raft_consensus.cc:385] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.281538 20517 raft_consensus.cc:740] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d5f54dc7d945c7b306d794eaa411bf, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.281443 20520 consensus_queue.cc:260] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.281778 20517 consensus_queue.cc:260] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.282274 20520 ts_tablet_manager.cc:1434] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:17:18.282567 20520 tablet_bootstrap.cc:492] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: Bootstrap starting.
I20251023 20:17:18.283576 20517 ts_tablet_manager.cc:1434] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:17:18.283812 20517 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: Bootstrap starting.
I20251023 20:17:18.284358 20520 tablet_bootstrap.cc:654] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.285234 20517 tablet_bootstrap.cc:654] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.290161 20521 tablet_bootstrap.cc:492] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822: No bootstrap required, opened a new log
I20251023 20:17:18.290169 20520 tablet_bootstrap.cc:492] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: No bootstrap required, opened a new log
I20251023 20:17:18.290273 20517 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: No bootstrap required, opened a new log
I20251023 20:17:18.290287 20521 ts_tablet_manager.cc:1403] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20251023 20:17:18.290287 20520 ts_tablet_manager.cc:1403] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.008s
I20251023 20:17:18.290395 20517 ts_tablet_manager.cc:1403] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20251023 20:17:18.290925 20520 raft_consensus.cc:359] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.290961 20517 raft_consensus.cc:359] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.290935 20521 raft_consensus.cc:359] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.291086 20520 raft_consensus.cc:385] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.291108 20517 raft_consensus.cc:385] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.291150 20520 raft_consensus.cc:740] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d123552770c4dccbf215da899937dee, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.291115 20521 raft_consensus.cc:385] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.291247 20521 raft_consensus.cc:740] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088a0d55fd23477b81facff331c2a822, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.291375 20520 consensus_queue.cc:260] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.291393 20521 consensus_queue.cc:260] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.291174 20517 raft_consensus.cc:740] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d5f54dc7d945c7b306d794eaa411bf, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.291841 20520 ts_tablet_manager.cc:1434] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251023 20:17:18.291787 20517 consensus_queue.cc:260] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.292066 20520 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: Bootstrap starting.
I20251023 20:17:18.292145 20517 ts_tablet_manager.cc:1434] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:17:18.292505 20524 raft_consensus.cc:493] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:18.292685 20524 raft_consensus.cc:515] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.293627 20520 tablet_bootstrap.cc:654] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.295075 20519 raft_consensus.cc:493] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:18.295195 20519 raft_consensus.cc:515] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.295962 20519 leader_election.cc:290] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d123552770c4dccbf215da899937dee (127.15.81.193:32959), 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
I20251023 20:17:18.309803 20524 leader_election.cc:290] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855), 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
I20251023 20:17:18.319511 20519 raft_consensus.cc:493] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:18.319679 20519 raft_consensus.cc:515] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.320325 20519 leader_election.cc:290] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d123552770c4dccbf215da899937dee (127.15.81.193:32959), 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
W20251023 20:17:18.327962 20349 tablet.cc:2378] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 20:17:18.334936 20521 ts_tablet_manager.cc:1434] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822: Time spent starting tablet: real 0.045s user 0.003s sys 0.008s
I20251023 20:17:18.335211 20521 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: Bootstrap starting.
I20251023 20:17:18.335275 20520 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: No bootstrap required, opened a new log
I20251023 20:17:18.335404 20520 ts_tablet_manager.cc:1403] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.031s
I20251023 20:17:18.336211 20520 raft_consensus.cc:359] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.336361 20520 raft_consensus.cc:385] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.336414 20520 raft_consensus.cc:740] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d123552770c4dccbf215da899937dee, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.336627 20520 consensus_queue.cc:260] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.336845 20398 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "6d123552770c4dccbf215da899937dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822" is_pre_election: true
I20251023 20:17:18.336951 20521 tablet_bootstrap.cc:654] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.337245 20520 ts_tablet_manager.cc:1434] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:17:18.337918 20398 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822" is_pre_election: true
W20251023 20:17:18.338550 20288 leader_election.cc:343] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:17:18.338842 20398 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81d51f1bd2494650914b71b074d058fb" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822" is_pre_election: true
W20251023 20:17:18.339440 20437 leader_election.cc:343] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973): Illegal state: must be running to vote when last-logged opid is not known
W20251023 20:17:18.339661 20437 leader_election.cc:343] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:17:18.340075 20323 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d123552770c4dccbf215da899937dee" is_pre_election: true
I20251023 20:17:18.340282 20323 raft_consensus.cc:2468] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d5f54dc7d945c7b306d794eaa411bf in term 0.
I20251023 20:17:18.340212 20322 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81d51f1bd2494650914b71b074d058fb" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d123552770c4dccbf215da899937dee" is_pre_election: true
I20251023 20:17:18.340852 20322 raft_consensus.cc:2468] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d5f54dc7d945c7b306d794eaa411bf in term 0.
I20251023 20:17:18.340988 20438 leader_election.cc:304] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d123552770c4dccbf215da899937dee, d5d5f54dc7d945c7b306d794eaa411bf; no voters: 088a0d55fd23477b81facff331c2a822
I20251023 20:17:18.341444 20519 raft_consensus.cc:2804] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:18.341531 20519 raft_consensus.cc:493] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:18.341604 20519 raft_consensus.cc:3060] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.342085 20438 leader_election.cc:304] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d123552770c4dccbf215da899937dee, d5d5f54dc7d945c7b306d794eaa411bf; no voters: 088a0d55fd23477b81facff331c2a822
I20251023 20:17:18.342610 20536 raft_consensus.cc:2804] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:18.342739 20536 raft_consensus.cc:493] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:18.342825 20472 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "6d123552770c4dccbf215da899937dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" is_pre_election: true
I20251023 20:17:18.342868 20536 raft_consensus.cc:3060] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.343304 20521 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: No bootstrap required, opened a new log
I20251023 20:17:18.343436 20521 ts_tablet_manager.cc:1403] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251023 20:17:18.344058 20521 raft_consensus.cc:359] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.344280 20521 raft_consensus.cc:385] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.344339 20521 raft_consensus.cc:740] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088a0d55fd23477b81facff331c2a822, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.344318 20519 raft_consensus.cc:515] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.344635 20472 raft_consensus.cc:2393] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6d123552770c4dccbf215da899937dee in current term 1: Already voted for candidate d5d5f54dc7d945c7b306d794eaa411bf in this term.
I20251023 20:17:18.344535 20521 consensus_queue.cc:260] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.344851 20519 leader_election.cc:290] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 election: Requested vote from peers 6d123552770c4dccbf215da899937dee (127.15.81.193:32959), 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
I20251023 20:17:18.345086 20521 ts_tablet_manager.cc:1434] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:17:18.345326 20521 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822: Bootstrap starting.
I20251023 20:17:18.345388 20286 leader_election.cc:304] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d123552770c4dccbf215da899937dee; no voters: 088a0d55fd23477b81facff331c2a822, d5d5f54dc7d945c7b306d794eaa411bf
I20251023 20:17:18.345664 20322 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d123552770c4dccbf215da899937dee"
I20251023 20:17:18.345769 20524 raft_consensus.cc:3060] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.346276 20398 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822"
I20251023 20:17:18.346422 20398 raft_consensus.cc:3060] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.346885 20521 tablet_bootstrap.cc:654] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:18.348207 20524 raft_consensus.cc:2749] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:17:18.348712 20521 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822: No bootstrap required, opened a new log
I20251023 20:17:18.348835 20521 ts_tablet_manager.cc:1403] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:17:18.349792 20521 raft_consensus.cc:359] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.349948 20521 raft_consensus.cc:385] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:18.350010 20521 raft_consensus.cc:740] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088a0d55fd23477b81facff331c2a822, State: Initialized, Role: FOLLOWER
I20251023 20:17:18.350277 20521 consensus_queue.cc:260] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.350847 20521 ts_tablet_manager.cc:1434] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:17:18.351372 20398 raft_consensus.cc:2468] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d5f54dc7d945c7b306d794eaa411bf in term 1.
I20251023 20:17:18.351692 20322 raft_consensus.cc:2468] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d5f54dc7d945c7b306d794eaa411bf in term 1.
I20251023 20:17:18.351743 20437 leader_election.cc:304] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 088a0d55fd23477b81facff331c2a822, d5d5f54dc7d945c7b306d794eaa411bf; no voters:
I20251023 20:17:18.351980 20519 raft_consensus.cc:2804] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:18.353083 20536 raft_consensus.cc:515] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.353293 20524 raft_consensus.cc:493] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:18.353435 20524 raft_consensus.cc:515] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.353646 20528 raft_consensus.cc:493] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:18.353756 20528 raft_consensus.cc:515] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.354070 20322 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81d51f1bd2494650914b71b074d058fb" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d123552770c4dccbf215da899937dee"
I20251023 20:17:18.354185 20524 leader_election.cc:290] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973), d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855)
I20251023 20:17:18.354213 20322 raft_consensus.cc:3060] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.354271 20398 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5de6d5c5a8f642379e215d09277b4224" candidate_uuid: "6d123552770c4dccbf215da899937dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822" is_pre_election: true
I20251023 20:17:18.354395 20398 raft_consensus.cc:2468] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d123552770c4dccbf215da899937dee in term 0.
I20251023 20:17:18.354686 20288 leader_election.cc:304] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 088a0d55fd23477b81facff331c2a822, 6d123552770c4dccbf215da899937dee; no voters:
I20251023 20:17:18.354683 20472 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5de6d5c5a8f642379e215d09277b4224" candidate_uuid: "6d123552770c4dccbf215da899937dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" is_pre_election: true
I20251023 20:17:18.354885 20472 raft_consensus.cc:2468] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d123552770c4dccbf215da899937dee in term 0.
I20251023 20:17:18.355337 20524 raft_consensus.cc:2804] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:18.355425 20524 raft_consensus.cc:493] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:18.355826 20524 raft_consensus.cc:3060] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.357307 20322 raft_consensus.cc:2468] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d5f54dc7d945c7b306d794eaa411bf in term 1.
I20251023 20:17:18.357647 20438 leader_election.cc:304] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d123552770c4dccbf215da899937dee, d5d5f54dc7d945c7b306d794eaa411bf; no voters:
I20251023 20:17:18.358796 20524 raft_consensus.cc:515] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.360383 20472 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5de6d5c5a8f642379e215d09277b4224" candidate_uuid: "6d123552770c4dccbf215da899937dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d5f54dc7d945c7b306d794eaa411bf"
I20251023 20:17:18.360567 20472 raft_consensus.cc:3060] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.363701 20472 raft_consensus.cc:2468] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d123552770c4dccbf215da899937dee in term 1.
I20251023 20:17:18.363794 20536 leader_election.cc:290] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [CANDIDATE]: Term 1 election: Requested vote from peers 6d123552770c4dccbf215da899937dee (127.15.81.193:32959), 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
I20251023 20:17:18.363903 20528 leader_election.cc:290] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d123552770c4dccbf215da899937dee (127.15.81.193:32959), d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855)
I20251023 20:17:18.363974 20397 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5de6d5c5a8f642379e215d09277b4224" candidate_uuid: "6d123552770c4dccbf215da899937dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822"
I20251023 20:17:18.364019 20536 raft_consensus.cc:2804] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:18.364104 20397 raft_consensus.cc:3060] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.364022 20286 leader_election.cc:304] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d123552770c4dccbf215da899937dee, d5d5f54dc7d945c7b306d794eaa411bf; no voters:
I20251023 20:17:18.364144 20536 raft_consensus.cc:697] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Becoming Leader. State: Replica: d5d5f54dc7d945c7b306d794eaa411bf, State: Running, Role: LEADER
I20251023 20:17:18.364355 20536 consensus_queue.cc:237] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.366995 20397 raft_consensus.cc:2468] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d123552770c4dccbf215da899937dee in term 1.
I20251023 20:17:18.367779 20398 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81d51f1bd2494650914b71b074d058fb" candidate_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088a0d55fd23477b81facff331c2a822"
I20251023 20:17:18.367937 20398 raft_consensus.cc:3060] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:18.370337 20524 leader_election.cc:290] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [CANDIDATE]: Term 1 election: Requested vote from peers 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973), d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855)
I20251023 20:17:18.370450 20542 raft_consensus.cc:2804] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:18.371165 20398 raft_consensus.cc:2468] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d5f54dc7d945c7b306d794eaa411bf in term 1.
I20251023 20:17:18.371253 20542 raft_consensus.cc:697] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 1 LEADER]: Becoming Leader. State: Replica: 6d123552770c4dccbf215da899937dee, State: Running, Role: LEADER
I20251023 20:17:18.371585 20542 consensus_queue.cc:237] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } }
I20251023 20:17:18.372467 20519 raft_consensus.cc:697] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Becoming Leader. State: Replica: d5d5f54dc7d945c7b306d794eaa411bf, State: Running, Role: LEADER
I20251023 20:17:18.372819 20519 consensus_queue.cc:237] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:18.379312 20220 catalog_manager.cc:5649] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d123552770c4dccbf215da899937dee (127.15.81.193). New cstate: current_term: 1 leader_uuid: "6d123552770c4dccbf215da899937dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:18.380800 20221 catalog_manager.cc:5649] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf reported cstate change: term changed from 0 to 1, leader changed from <none> to d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195). New cstate: current_term: 1 leader_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:18.385085 20221 catalog_manager.cc:5649] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf reported cstate change: term changed from 0 to 1, leader changed from <none> to d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195). New cstate: current_term: 1 leader_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } health_report { overall_health: HEALTHY } } }
I20251023 20:17:18.387526 20322 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5de6d5c5a8f642379e215d09277b4224" candidate_uuid: "088a0d55fd23477b81facff331c2a822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d123552770c4dccbf215da899937dee" is_pre_election: true
I20251023 20:17:18.388293 20472 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5de6d5c5a8f642379e215d09277b4224" candidate_uuid: "088a0d55fd23477b81facff331c2a822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" is_pre_election: true
I20251023 20:17:18.388538 20472 raft_consensus.cc:2393] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 088a0d55fd23477b81facff331c2a822 in current term 1: Already voted for candidate 6d123552770c4dccbf215da899937dee in this term.
I20251023 20:17:18.388924 20361 leader_election.cc:304] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 088a0d55fd23477b81facff331c2a822; no voters: 6d123552770c4dccbf215da899937dee, d5d5f54dc7d945c7b306d794eaa411bf
I20251023 20:17:18.389189 20528 raft_consensus.cc:2749] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:17:18.416992 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:18.419456 20546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:18.419478 20547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:18.421197 20549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:18.421486 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:18.421671 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:18.421728 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:18.421769 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250638421756 us; error 0 us; skew 500 ppm
I20251023 20:17:18.421883 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:18.422930 15687 webserver.cc:492] Webserver started at http://127.15.81.196:44741/ using document root <none> and password file <none>
I20251023 20:17:18.423122 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:18.423183 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:18.423285 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:18.423801 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "17c4b18be5e24be78bf51d1ce5e2328f"
format_stamp: "Formatted at 2025-10-23 20:17:18 on dist-test-slave-jr94"
I20251023 20:17:18.426093 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:18.427536 20554 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:18.427806 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:18.427906 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "17c4b18be5e24be78bf51d1ce5e2328f"
format_stamp: "Formatted at 2025-10-23 20:17:18 on dist-test-slave-jr94"
I20251023 20:17:18.428004 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:18.436658 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:18.437098 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:18.437553 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:18.438272 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:18.438345 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:18.438458 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:18.438514 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:18.450002 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:35169
I20251023 20:17:18.450026 20616 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:35169 every 8 connection(s)
I20251023 20:17:18.456176 20617 heartbeater.cc:344] Connected to a master server at 127.15.81.254:45449
I20251023 20:17:18.456282 20617 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:18.456561 20617 heartbeater.cc:507] Master 127.15.81.254:45449 requested a full tablet report, sending...
I20251023 20:17:18.457245 20221 ts_manager.cc:194] Registered new tserver with Master: 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196:35169)
I20251023 20:17:18.457850 20221 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46112
I20251023 20:17:18.469609 20542 consensus_queue.cc:1048] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:17:18.471707 20524 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20251023 20:17:18.471804 20524 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20251023 20:17:18.473937 20541 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:18.478394 20541 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:18.480424 20536 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:18.495015 20536 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:18.496349 20542 consensus_queue.cc:1048] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:19.000044 20274 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:19.005769 20472 consensus_queue.cc:237] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } }
I20251023 20:17:19.007050 20471 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df"
dest_uuid: "d5d5f54dc7d945c7b306d794eaa411bf"
mode: GRACEFUL
new_leader_uuid: "088a0d55fd23477b81facff331c2a822"
from {username='slave'} at 127.0.0.1:60712
I20251023 20:17:19.007236 20471 raft_consensus.cc:606] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Received request to transfer leadership to TS 088a0d55fd23477b81facff331c2a822
I20251023 20:17:19.007654 20274 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251023 20:17:19.007836 20274 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:19.007972 20322 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:17:19.008283 20398 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:17:19.008714 20536 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:17:19.009263 20538 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:17:19.011780 20322 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
I20251023 20:17:19.012209 20398 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
W20251023 20:17:19.017251 20436 consensus_peers.cc:597] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf -> Peer 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196:35169): Couldn't send request to peer 17c4b18be5e24be78bf51d1ce5e2328f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81d51f1bd2494650914b71b074d058fb. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:19.017228 20220 catalog_manager.cc:5649] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 reported cstate change: config changed from index -1 to 2, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
I20251023 20:17:19.054236 20536 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
W20251023 20:17:19.058045 20269 auto_rebalancer.cc:254] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
W20251023 20:17:19.060267 20269 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 71cf935bdaff41f5a0f284b97b9e69df, TS 088a0d55fd23477b81facff331c2a822 -> TS 17c4b18be5e24be78bf51d1ce5e2328f move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
W20251023 20:17:19.060354 20269 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 71cf935bdaff41f5a0f284b97b9e69df, TS 088a0d55fd23477b81facff331c2a822 -> TS 17c4b18be5e24be78bf51d1ce5e2328f move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:19.060992 20538 raft_consensus.cc:993] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: : Instructing follower 088a0d55fd23477b81facff331c2a822 to start an election
I20251023 20:17:19.061190 20541 raft_consensus.cc:1081] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Signalling peer 088a0d55fd23477b81facff331c2a822 to start an election
I20251023 20:17:19.061745 20398 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df"
dest_uuid: "088a0d55fd23477b81facff331c2a822"
from {username='slave'} at 127.0.0.1:44854
I20251023 20:17:19.061926 20398 raft_consensus.cc:493] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251023 20:17:19.062059 20398 raft_consensus.cc:3060] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:19.064148 20398 raft_consensus.cc:515] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:19.064791 20398 leader_election.cc:290] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [CANDIDATE]: Term 2 election: Requested vote from peers d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855), 6d123552770c4dccbf215da899937dee (127.15.81.193:32959)
I20251023 20:17:19.065141 20471 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "088a0d55fd23477b81facff331c2a822" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5d5f54dc7d945c7b306d794eaa411bf"
I20251023 20:17:19.065308 20322 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71cf935bdaff41f5a0f284b97b9e69df" candidate_uuid: "088a0d55fd23477b81facff331c2a822" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d123552770c4dccbf215da899937dee"
I20251023 20:17:19.065513 20322 raft_consensus.cc:3060] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:19.065310 20471 raft_consensus.cc:3055] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Stepping down as leader of term 1
I20251023 20:17:19.065659 20471 raft_consensus.cc:740] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d5d5f54dc7d945c7b306d794eaa411bf, State: Running, Role: LEADER
I20251023 20:17:19.065948 20471 consensus_queue.cc:260] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:19.066287 20471 raft_consensus.cc:3060] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:19.067737 20322 raft_consensus.cc:2468] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 088a0d55fd23477b81facff331c2a822 in term 2.
I20251023 20:17:19.068076 20364 leader_election.cc:304] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 088a0d55fd23477b81facff331c2a822, 6d123552770c4dccbf215da899937dee; no voters:
I20251023 20:17:19.068322 20636 raft_consensus.cc:2804] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:17:19.068765 20636 raft_consensus.cc:697] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 LEADER]: Becoming Leader. State: Replica: 088a0d55fd23477b81facff331c2a822, State: Running, Role: LEADER
I20251023 20:17:19.068830 20471 raft_consensus.cc:2468] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 088a0d55fd23477b81facff331c2a822 in term 2.
I20251023 20:17:19.069067 20636 consensus_queue.cc:237] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } }
I20251023 20:17:19.089305 20640 ts_tablet_manager.cc:933] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: Initiating tablet copy from peer d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855)
I20251023 20:17:19.089943 20640 tablet_copy_client.cc:323] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:34855
I20251023 20:17:19.095376 20482 tablet_copy_service.cc:140] P d5d5f54dc7d945c7b306d794eaa411bf: Received BeginTabletCopySession request for tablet 81d51f1bd2494650914b71b074d058fb from peer 17c4b18be5e24be78bf51d1ce5e2328f ({username='slave'} at 127.0.0.1:60724)
I20251023 20:17:19.095556 20482 tablet_copy_service.cc:161] P d5d5f54dc7d945c7b306d794eaa411bf: Beginning new tablet copy session on tablet 81d51f1bd2494650914b71b074d058fb from peer 17c4b18be5e24be78bf51d1ce5e2328f at {username='slave'} at 127.0.0.1:60724: session id = 17c4b18be5e24be78bf51d1ce5e2328f-81d51f1bd2494650914b71b074d058fb
I20251023 20:17:19.097743 20482 tablet_copy_source_session.cc:215] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:17:19.098853 20640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d51f1bd2494650914b71b074d058fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:19.103451 20640 tablet_copy_client.cc:806] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Starting download of 0 data blocks...
I20251023 20:17:19.103701 20640 tablet_copy_client.cc:670] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Starting download of 1 WAL segments...
I20251023 20:17:19.105082 20640 tablet_copy_client.cc:538] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:19.107805 20640 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: Bootstrap starting.
I20251023 20:17:19.113029 20640 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:19.113407 20640 tablet_bootstrap.cc:492] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: Bootstrap complete.
I20251023 20:17:19.113667 20640 ts_tablet_manager.cc:1403] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.002s
I20251023 20:17:19.114442 20640 raft_consensus.cc:359] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } }
I20251023 20:17:19.114677 20640 raft_consensus.cc:740] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 17c4b18be5e24be78bf51d1ce5e2328f, State: Initialized, Role: LEARNER
I20251023 20:17:19.114871 20640 consensus_queue.cc:260] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } }
I20251023 20:17:19.115581 20617 heartbeater.cc:499] Master 127.15.81.254:45449 was elected leader, sending a full tablet report...
I20251023 20:17:19.115598 20640 ts_tablet_manager.cc:1434] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:17:19.116297 20482 tablet_copy_service.cc:342] P d5d5f54dc7d945c7b306d794eaa411bf: Request end of tablet copy session 17c4b18be5e24be78bf51d1ce5e2328f-81d51f1bd2494650914b71b074d058fb received from {username='slave'} at 127.0.0.1:60724
I20251023 20:17:19.116416 20482 tablet_copy_service.cc:434] P d5d5f54dc7d945c7b306d794eaa411bf: ending tablet copy session 17c4b18be5e24be78bf51d1ce5e2328f-81d51f1bd2494650914b71b074d058fb on tablet 81d51f1bd2494650914b71b074d058fb with peer 17c4b18be5e24be78bf51d1ce5e2328f
I20251023 20:17:19.120455 20221 catalog_manager.cc:5649] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 reported cstate change: term changed from 1 to 2, leader changed from d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195) to 088a0d55fd23477b81facff331c2a822 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "088a0d55fd23477b81facff331c2a822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } health_report { overall_health: HEALTHY } } }
I20251023 20:17:19.154829 20322 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:17:19.154829 20471 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:17:19.155319 20639 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:17:19.155565 20636 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:17:19.181612 20592 raft_consensus.cc:1217] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 20:17:19.324774 20536 raft_consensus.cc:1064] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: attempting to promote NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f to VOTER
I20251023 20:17:19.325543 20536 consensus_queue.cc:237] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } }
I20251023 20:17:19.327205 20592 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 LEARNER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:19.327306 20322 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:19.327303 20398 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:19.327759 20541 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:19.328003 20538 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:19.328226 20519 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:19.331316 20398 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.331574 20592 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.335691 20221 catalog_manager.cc:5649] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f reported cstate change: config changed from index 2 to 3, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.394754 20519 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.398309 20322 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } attrs { replace: true } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.400772 20472 consensus_queue.cc:237] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } }
I20251023 20:17:19.402462 20592 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 FOLLOWER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:19.402565 20398 raft_consensus.cc:1275] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Refusing update from remote peer d5d5f54dc7d945c7b306d794eaa411bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:19.402947 20538 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:19.403187 20651 consensus_queue.cc:1048] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:19.406445 20398 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6d123552770c4dccbf215da899937dee (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.405725 20592 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6d123552770c4dccbf215da899937dee (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.412531 20218 catalog_manager.cc:5649] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 reported cstate change: config changed from index 3 to 4, VOTER 6d123552770c4dccbf215da899937dee (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.481736 20651 raft_consensus.cc:2955] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6d123552770c4dccbf215da899937dee (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:19.484143 20205 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 81d51f1bd2494650914b71b074d058fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:19.513151 20312 tablet_service.cc:1552] Processing DeleteTablet for tablet 81d51f1bd2494650914b71b074d058fb with delete_type TABLET_DATA_TOMBSTONED (TS 6d123552770c4dccbf215da899937dee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41574
I20251023 20:17:19.513895 20652 tablet_replica.cc:333] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: stopping tablet replica
I20251023 20:17:19.514096 20652 raft_consensus.cc:2243] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:19.514328 20652 raft_consensus.cc:2272] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:19.515010 20652 ts_tablet_manager.cc:1916] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:17:19.519162 20652 ts_tablet_manager.cc:1929] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251023 20:17:19.519246 20652 log.cc:1199] T 81d51f1bd2494650914b71b074d058fb P 6d123552770c4dccbf215da899937dee: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/81d51f1bd2494650914b71b074d058fb
I20251023 20:17:19.519824 20208 catalog_manager.cc:4985] TS 6d123552770c4dccbf215da899937dee (127.15.81.193:32959): tablet 81d51f1bd2494650914b71b074d058fb (table test-workload [id=e1d7ddb95f50475b852939e01ea26256]) successfully deleted
I20251023 20:17:20.492353 20398 consensus_queue.cc:237] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } }
I20251023 20:17:20.495213 20322 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:17:20.495674 20471 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:17:20.495750 20639 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:20.496073 20658 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:20.498944 20471 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
I20251023 20:17:20.498931 20322 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
W20251023 20:17:20.502101 20362 consensus_peers.cc:597] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 -> Peer 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196:35169): Couldn't send request to peer 17c4b18be5e24be78bf51d1ce5e2328f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71cf935bdaff41f5a0f284b97b9e69df. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:20.503618 20220 catalog_manager.cc:5649] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf reported cstate change: config changed from index -1 to 3, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New cstate: current_term: 2 leader_uuid: "088a0d55fd23477b81facff331c2a822" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
I20251023 20:17:20.559470 20656 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } } }
I20251023 20:17:20.609164 20666 ts_tablet_manager.cc:933] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: Initiating tablet copy from peer 088a0d55fd23477b81facff331c2a822 (127.15.81.194:43973)
I20251023 20:17:20.609853 20666 tablet_copy_client.cc:323] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:43973
I20251023 20:17:20.615794 20408 tablet_copy_service.cc:140] P 088a0d55fd23477b81facff331c2a822: Received BeginTabletCopySession request for tablet 71cf935bdaff41f5a0f284b97b9e69df from peer 17c4b18be5e24be78bf51d1ce5e2328f ({username='slave'} at 127.0.0.1:44882)
I20251023 20:17:20.615908 20408 tablet_copy_service.cc:161] P 088a0d55fd23477b81facff331c2a822: Beginning new tablet copy session on tablet 71cf935bdaff41f5a0f284b97b9e69df from peer 17c4b18be5e24be78bf51d1ce5e2328f at {username='slave'} at 127.0.0.1:44882: session id = 17c4b18be5e24be78bf51d1ce5e2328f-71cf935bdaff41f5a0f284b97b9e69df
I20251023 20:17:20.617756 20408 tablet_copy_source_session.cc:215] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:17:20.618752 20666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cf935bdaff41f5a0f284b97b9e69df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:20.623219 20666 tablet_copy_client.cc:806] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Starting download of 0 data blocks...
I20251023 20:17:20.623413 20666 tablet_copy_client.cc:670] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Starting download of 1 WAL segments...
W20251023 20:17:20.623903 20613 debug-util.cc:398] Leaking SignalData structure 0x6030013ab300 after lost signal to thread 20643
I20251023 20:17:20.625021 20666 tablet_copy_client.cc:538] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:20.626893 20666 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: Bootstrap starting.
I20251023 20:17:20.635758 20666 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:20.636171 20666 tablet_bootstrap.cc:492] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: Bootstrap complete.
I20251023 20:17:20.636412 20666 ts_tablet_manager.cc:1403] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20251023 20:17:20.637254 20666 raft_consensus.cc:359] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } }
I20251023 20:17:20.637485 20666 raft_consensus.cc:740] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 17c4b18be5e24be78bf51d1ce5e2328f, State: Initialized, Role: LEARNER
I20251023 20:17:20.637677 20666 consensus_queue.cc:260] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: true } }
I20251023 20:17:20.648947 20666 ts_tablet_manager.cc:1434] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: Time spent starting tablet: real 0.012s user 0.014s sys 0.001s
I20251023 20:17:20.650292 20408 tablet_copy_service.cc:342] P 088a0d55fd23477b81facff331c2a822: Request end of tablet copy session 17c4b18be5e24be78bf51d1ce5e2328f-71cf935bdaff41f5a0f284b97b9e69df received from {username='slave'} at 127.0.0.1:44882
I20251023 20:17:20.650384 20408 tablet_copy_service.cc:434] P 088a0d55fd23477b81facff331c2a822: ending tablet copy session 17c4b18be5e24be78bf51d1ce5e2328f-71cf935bdaff41f5a0f284b97b9e69df on tablet 71cf935bdaff41f5a0f284b97b9e69df with peer 17c4b18be5e24be78bf51d1ce5e2328f
I20251023 20:17:20.696012 20592 raft_consensus.cc:1217] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251023 20:17:20.861479 20658 raft_consensus.cc:1064] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: attempting to promote NON_VOTER 17c4b18be5e24be78bf51d1ce5e2328f to VOTER
I20251023 20:17:20.862506 20658 consensus_queue.cc:237] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } }
I20251023 20:17:20.864591 20471 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:17:20.864667 20322 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:17:20.864748 20592 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 LEARNER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:17:20.865149 20656 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:20.865336 20639 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:20.865521 20664 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:20.868223 20322 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.872184 20592 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.873082 20218 catalog_manager.cc:5649] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee reported cstate change: config changed from index 3 to 4, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "088a0d55fd23477b81facff331c2a822" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.874866 20656 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.878927 20471 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 17c4b18be5e24be78bf51d1ce5e2328f (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5d5f54dc7d945c7b306d794eaa411bf" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.930444 20397 consensus_queue.cc:237] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } }
I20251023 20:17:20.932258 20322 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 20:17:20.932673 20592 raft_consensus.cc:1275] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 FOLLOWER]: Refusing update from remote peer 088a0d55fd23477b81facff331c2a822: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 20:17:20.932833 20658 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:17:20.933187 20664 consensus_queue.cc:1048] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:17:20.935958 20592 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.936107 20322 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:20.939338 20221 catalog_manager.cc:5649] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f reported cstate change: config changed from index 4 to 5, VOTER d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195) evicted. New cstate: current_term: 2 leader_uuid: "088a0d55fd23477b81facff331c2a822" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:21.008188 20274 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:21.009620 20461 tablet_service.cc:1552] Processing DeleteTablet for tablet 71cf935bdaff41f5a0f284b97b9e69df with delete_type TABLET_DATA_TOMBSTONED (TS d5d5f54dc7d945c7b306d794eaa411bf not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60674
I20251023 20:17:21.009768 20274 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:21.009882 20274 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:21.010795 20672 tablet_replica.cc:333] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: stopping tablet replica
I20251023 20:17:21.010973 20672 raft_consensus.cc:2243] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:21.011199 20672 raft_consensus.cc:2272] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:21.011886 20672 ts_tablet_manager.cc:1916] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:17:21.016465 20672 ts_tablet_manager.cc:1929] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251023 20:17:21.016582 20672 log.cc:1199] T 71cf935bdaff41f5a0f284b97b9e69df P d5d5f54dc7d945c7b306d794eaa411bf: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/71cf935bdaff41f5a0f284b97b9e69df
I20251023 20:17:21.017185 20205 catalog_manager.cc:4985] TS d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195:34855): tablet 71cf935bdaff41f5a0f284b97b9e69df (table test-workload [id=e1d7ddb95f50475b852939e01ea26256]) successfully deleted
I20251023 20:17:21.043912 20658 raft_consensus.cc:2955] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d5d5f54dc7d945c7b306d794eaa411bf (127.15.81.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6d123552770c4dccbf215da899937dee" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 32959 } } peers { permanent_uuid: "088a0d55fd23477b81facff331c2a822" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43973 } } peers { permanent_uuid: "17c4b18be5e24be78bf51d1ce5e2328f" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 35169 } attrs { promote: false } } }
I20251023 20:17:21.046865 20207 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 71cf935bdaff41f5a0f284b97b9e69df with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:23.010254 20274 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:23.010885 20274 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:23.010995 20274 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:25.011413 20274 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:25.012079 20274 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:25.012185 20274 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:27.012576 20274 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:27.013236 20274 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:27.013336 20274 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:27.523779 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
W20251023 20:17:27.533782 20364 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20251023 20:17:27.540424 20364 consensus_peers.cc:597] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 -> Peer 6d123552770c4dccbf215da899937dee (127.15.81.193:32959): Couldn't send request to peer 6d123552770c4dccbf215da899937dee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:27.554332 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:27.554579 15687 tablet_replica.cc:333] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee: stopping tablet replica
I20251023 20:17:27.554756 15687 raft_consensus.cc:2243] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:27.554976 15687 raft_consensus.cc:2272] T 71cf935bdaff41f5a0f284b97b9e69df P 6d123552770c4dccbf215da899937dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.555502 15687 tablet_replica.cc:333] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee: stopping tablet replica
I20251023 20:17:27.555634 15687 raft_consensus.cc:2243] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:27.555860 15687 raft_consensus.cc:2272] T 5de6d5c5a8f642379e215d09277b4224 P 6d123552770c4dccbf215da899937dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.559952 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:17:27.563182 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:17:27.573941 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:27.574165 15687 tablet_replica.cc:333] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822: stopping tablet replica
I20251023 20:17:27.574326 15687 raft_consensus.cc:2243] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:27.574553 15687 raft_consensus.cc:2272] T 81d51f1bd2494650914b71b074d058fb P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.575023 15687 tablet_replica.cc:333] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822: stopping tablet replica
I20251023 20:17:27.575165 15687 raft_consensus.cc:2243] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:27.575281 15687 raft_consensus.cc:2272] T 5de6d5c5a8f642379e215d09277b4224 P 088a0d55fd23477b81facff331c2a822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.575671 15687 tablet_replica.cc:333] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822: stopping tablet replica
I20251023 20:17:27.575780 15687 raft_consensus.cc:2243] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:27.576020 15687 raft_consensus.cc:2272] T 71cf935bdaff41f5a0f284b97b9e69df P 088a0d55fd23477b81facff331c2a822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.589967 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:17:27.592763 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:17:27.601490 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:27.601675 15687 tablet_replica.cc:333] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf: stopping tablet replica
I20251023 20:17:27.601837 15687 raft_consensus.cc:2243] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:27.602093 15687 raft_consensus.cc:2272] T 81d51f1bd2494650914b71b074d058fb P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.602540 15687 tablet_replica.cc:333] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf: stopping tablet replica
I20251023 20:17:27.602653 15687 raft_consensus.cc:2243] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:27.602762 15687 raft_consensus.cc:2272] T 5de6d5c5a8f642379e215d09277b4224 P d5d5f54dc7d945c7b306d794eaa411bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.618043 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:17:27.621145 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:17:27.629017 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:27.629289 15687 tablet_replica.cc:333] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f: stopping tablet replica
I20251023 20:17:27.629472 15687 raft_consensus.cc:2243] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:27.629671 15687 raft_consensus.cc:2272] T 71cf935bdaff41f5a0f284b97b9e69df P 17c4b18be5e24be78bf51d1ce5e2328f [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.630155 15687 tablet_replica.cc:333] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f: stopping tablet replica
I20251023 20:17:27.630265 15687 raft_consensus.cc:2243] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:27.630434 15687 raft_consensus.cc:2272] T 81d51f1bd2494650914b71b074d058fb P 17c4b18be5e24be78bf51d1ce5e2328f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.644294 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:17:27.663021 15687 master.cc:561] Master@127.15.81.254:45449 shutting down...
I20251023 20:17:27.669896 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:27.670123 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:27.670229 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P b45efb26b0f9484a87eec770f30d8500: stopping tablet replica
I20251023 20:17:27.683137 15687 master.cc:583] Master@127.15.81.254:45449 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingUnstableCluster (10745 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor
I20251023 20:17:27.694608 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:32773
I20251023 20:17:27.695011 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:27.697389 20679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:27.697438 20677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:27.697448 15687 server_base.cc:1047] running on GCE node
W20251023 20:17:27.697403 20676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:27.698063 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:27.698136 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:27.698169 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250647698166 us; error 0 us; skew 500 ppm
I20251023 20:17:27.698285 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:27.699230 15687 webserver.cc:492] Webserver started at http://127.15.81.254:44793/ using document root <none> and password file <none>
I20251023 20:17:27.699453 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:27.699514 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:27.699625 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:27.700201 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "f2329099e07f4ee5ac9ba8b337af2454"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.702752 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:17:27.704128 20684 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.704468 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:17:27.704542 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "f2329099e07f4ee5ac9ba8b337af2454"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.704627 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:27.717628 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:27.718014 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:27.728403 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:32773
I20251023 20:17:27.728413 20735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:32773 every 8 connection(s)
I20251023 20:17:27.729489 20736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:27.733157 20736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454: Bootstrap starting.
I20251023 20:17:27.734301 20736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:27.735649 20736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454: No bootstrap required, opened a new log
I20251023 20:17:27.736146 20736 raft_consensus.cc:359] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2329099e07f4ee5ac9ba8b337af2454" member_type: VOTER }
I20251023 20:17:27.736274 20736 raft_consensus.cc:385] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:27.736326 20736 raft_consensus.cc:740] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2329099e07f4ee5ac9ba8b337af2454, State: Initialized, Role: FOLLOWER
I20251023 20:17:27.736521 20736 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2329099e07f4ee5ac9ba8b337af2454" member_type: VOTER }
I20251023 20:17:27.736639 20736 raft_consensus.cc:399] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:17:27.736717 20736 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:17:27.736806 20736 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:27.738389 20736 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2329099e07f4ee5ac9ba8b337af2454" member_type: VOTER }
I20251023 20:17:27.738569 20736 leader_election.cc:304] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2329099e07f4ee5ac9ba8b337af2454; no voters:
I20251023 20:17:27.738916 20736 leader_election.cc:290] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:17:27.739017 20739 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:27.739459 20739 raft_consensus.cc:697] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 1 LEADER]: Becoming Leader. State: Replica: f2329099e07f4ee5ac9ba8b337af2454, State: Running, Role: LEADER
I20251023 20:17:27.739671 20739 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2329099e07f4ee5ac9ba8b337af2454" member_type: VOTER }
I20251023 20:17:27.739737 20736 sys_catalog.cc:565] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:17:27.740289 20740 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2329099e07f4ee5ac9ba8b337af2454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2329099e07f4ee5ac9ba8b337af2454" member_type: VOTER } }
I20251023 20:17:27.740317 20741 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2329099e07f4ee5ac9ba8b337af2454. Latest consensus state: current_term: 1 leader_uuid: "f2329099e07f4ee5ac9ba8b337af2454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2329099e07f4ee5ac9ba8b337af2454" member_type: VOTER } }
I20251023 20:17:27.740489 20740 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:27.740623 20741 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:27.745028 20755 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20251023 20:17:27.745051 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:17:27.745255 20744 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:17:27.746287 20744 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:17:27.748559 20744 catalog_manager.cc:1357] Generated new cluster ID: 6248061e70d645e3aaab8231be537448
I20251023 20:17:27.748638 20744 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:17:27.755574 20744 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:17:27.756217 20744 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:17:27.771519 20744 catalog_manager.cc:6022] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454: Generated new TSK 0
I20251023 20:17:27.771713 20744 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:17:27.777537 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:27.779892 20757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:27.780230 20758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:27.780388 20760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:27.780901 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:27.781162 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:27.781226 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:27.781268 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250647781254 us; error 0 us; skew 500 ppm
I20251023 20:17:27.781392 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:27.782421 15687 webserver.cc:492] Webserver started at http://127.15.81.193:36105/ using document root <none> and password file <none>
I20251023 20:17:27.782601 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:27.782660 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:27.782755 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:27.783283 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "6503e150f4a04feba7e1331d66c80ac9"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.785567 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 20:17:27.786859 20765 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.787132 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:27.787228 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "6503e150f4a04feba7e1331d66c80ac9"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.787331 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:27.800940 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:27.801375 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:27.801851 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:27.802614 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:27.802680 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.802778 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:27.802829 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.813409 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:43589
I20251023 20:17:27.813431 20827 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:43589 every 8 connection(s)
I20251023 20:17:27.814882 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:27.817433 20832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:27.817874 20833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:27.819443 15687 server_base.cc:1047] running on GCE node
W20251023 20:17:27.819603 20835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:27.819947 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:27.820034 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:27.820104 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250647820079 us; error 0 us; skew 500 ppm
I20251023 20:17:27.820271 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:27.821431 15687 webserver.cc:492] Webserver started at http://127.15.81.194:45725/ using document root <none> and password file <none>
I20251023 20:17:27.821645 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:27.821724 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:27.821851 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:27.821914 20828 heartbeater.cc:344] Connected to a master server at 127.15.81.254:32773
I20251023 20:17:27.822049 20828 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:27.822374 20828 heartbeater.cc:507] Master 127.15.81.254:32773 requested a full tablet report, sending...
I20251023 20:17:27.822525 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "916509046ff043929d808dbe83dbfbaa"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.823139 20701 ts_manager.cc:194] Registered new tserver with Master: 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589)
I20251023 20:17:27.824098 20701 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47122
I20251023 20:17:27.824883 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:17:27.826300 20840 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.826583 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:17:27.826684 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "916509046ff043929d808dbe83dbfbaa"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.826799 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:27.845219 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:27.845625 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:27.846076 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:27.846779 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:27.846846 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.846935 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:27.846997 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.858187 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:40351
I20251023 20:17:27.858210 20902 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:40351 every 8 connection(s)
I20251023 20:17:27.859596 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:27.862198 20907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:27.862865 20906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:27.863299 20909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:27.863806 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:27.864051 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:27.864123 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:27.864182 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250647864160 us; error 0 us; skew 500 ppm
I20251023 20:17:27.864326 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:27.865553 15687 webserver.cc:492] Webserver started at http://127.15.81.195:39903/ using document root <none> and password file <none>
I20251023 20:17:27.865803 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:27.865906 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:27.865999 20903 heartbeater.cc:344] Connected to a master server at 127.15.81.254:32773
I20251023 20:17:27.866048 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:27.866130 20903 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:27.866428 20903 heartbeater.cc:507] Master 127.15.81.254:32773 requested a full tablet report, sending...
I20251023 20:17:27.866659 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "e9f02e72d02746559133ee5aa66e613c"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.867223 20701 ts_manager.cc:194] Registered new tserver with Master: 916509046ff043929d808dbe83dbfbaa (127.15.81.194:40351)
I20251023 20:17:27.867851 20701 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47130
I20251023 20:17:27.868991 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:17:27.870369 20914 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.870694 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:27.870792 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "e9f02e72d02746559133ee5aa66e613c"
format_stamp: "Formatted at 2025-10-23 20:17:27 on dist-test-slave-jr94"
I20251023 20:17:27.870888 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:27.881847 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:27.882263 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:27.882750 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:27.883411 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:27.883476 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.883562 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:27.883617 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:27.894759 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:44181
I20251023 20:17:27.894773 20976 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:44181 every 8 connection(s)
I20251023 20:17:27.902798 20977 heartbeater.cc:344] Connected to a master server at 127.15.81.254:32773
I20251023 20:17:27.902907 20977 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:27.903174 20977 heartbeater.cc:507] Master 127.15.81.254:32773 requested a full tablet report, sending...
I20251023 20:17:27.903846 20701 ts_manager.cc:194] Registered new tserver with Master: e9f02e72d02746559133ee5aa66e613c (127.15.81.195:44181)
I20251023 20:17:27.904016 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.008026747s
I20251023 20:17:27.904506 20701 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47138
I20251023 20:17:28.825309 20828 heartbeater.cc:499] Master 127.15.81.254:32773 was elected leader, sending a full tablet report...
I20251023 20:17:28.868981 20903 heartbeater.cc:499] Master 127.15.81.254:32773 was elected leader, sending a full tablet report...
I20251023 20:17:28.905616 20977 heartbeater.cc:499] Master 127.15.81.254:32773 was elected leader, sending a full tablet report...
I20251023 20:17:28.931202 15687 test_util.cc:276] Using random seed: 524958565
I20251023 20:17:28.943289 20701 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47148:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:17:28.943779 20701 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:17:28.979589 20868 tablet_service.cc:1505] Processing CreateTablet for tablet 674c6a7ef701404782d4d90a6323b549 (DEFAULT_TABLE table=test-workload [id=bdbddc0aee854bde97730a55009f40ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:17:28.980069 20868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674c6a7ef701404782d4d90a6323b549. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:28.980783 20942 tablet_service.cc:1505] Processing CreateTablet for tablet 674c6a7ef701404782d4d90a6323b549 (DEFAULT_TABLE table=test-workload [id=bdbddc0aee854bde97730a55009f40ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:17:28.981194 20942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674c6a7ef701404782d4d90a6323b549. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:28.983628 20793 tablet_service.cc:1505] Processing CreateTablet for tablet 674c6a7ef701404782d4d90a6323b549 (DEFAULT_TABLE table=test-workload [id=bdbddc0aee854bde97730a55009f40ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 20:17:28.983935 20793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674c6a7ef701404782d4d90a6323b549. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:28.986866 20999 tablet_bootstrap.cc:492] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c: Bootstrap starting.
I20251023 20:17:28.987537 20998 tablet_bootstrap.cc:492] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa: Bootstrap starting.
I20251023 20:17:28.988526 21000 tablet_bootstrap.cc:492] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: Bootstrap starting.
I20251023 20:17:28.988704 20999 tablet_bootstrap.cc:654] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:28.989171 20998 tablet_bootstrap.cc:654] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:28.990428 21000 tablet_bootstrap.cc:654] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:28.992580 21000 tablet_bootstrap.cc:492] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: No bootstrap required, opened a new log
I20251023 20:17:28.992684 20998 tablet_bootstrap.cc:492] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa: No bootstrap required, opened a new log
I20251023 20:17:28.992743 21000 ts_tablet_manager.cc:1403] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 20:17:28.992802 20998 ts_tablet_manager.cc:1403] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20251023 20:17:28.993270 20999 tablet_bootstrap.cc:492] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c: No bootstrap required, opened a new log
I20251023 20:17:28.993403 20999 ts_tablet_manager.cc:1403] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20251023 20:17:28.993486 21000 raft_consensus.cc:359] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:28.993486 20998 raft_consensus.cc:359] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:28.993722 21000 raft_consensus.cc:385] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:28.993726 20998 raft_consensus.cc:385] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:28.993827 20998 raft_consensus.cc:740] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 916509046ff043929d808dbe83dbfbaa, State: Initialized, Role: FOLLOWER
I20251023 20:17:28.993827 21000 raft_consensus.cc:740] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6503e150f4a04feba7e1331d66c80ac9, State: Initialized, Role: FOLLOWER
I20251023 20:17:28.994029 20999 raft_consensus.cc:359] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:28.994087 20998 consensus_queue.cc:260] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:28.994089 21000 consensus_queue.cc:260] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:28.994196 20999 raft_consensus.cc:385] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:28.994371 20999 raft_consensus.cc:740] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9f02e72d02746559133ee5aa66e613c, State: Initialized, Role: FOLLOWER
I20251023 20:17:28.994710 20999 consensus_queue.cc:260] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:28.994798 20998 ts_tablet_manager.cc:1434] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:17:28.996071 21000 ts_tablet_manager.cc:1434] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:17:28.996865 20999 ts_tablet_manager.cc:1434] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:17:29.348974 21005 raft_consensus.cc:493] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:29.349167 21005 raft_consensus.cc:515] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:29.349970 21005 leader_election.cc:290] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 916509046ff043929d808dbe83dbfbaa (127.15.81.194:40351), e9f02e72d02746559133ee5aa66e613c (127.15.81.195:44181)
I20251023 20:17:29.352629 21006 raft_consensus.cc:493] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:29.352823 21006 raft_consensus.cc:515] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:29.353612 21006 leader_election.cc:290] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589), 916509046ff043929d808dbe83dbfbaa (127.15.81.194:40351)
I20251023 20:17:29.360461 20878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "6503e150f4a04feba7e1331d66c80ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916509046ff043929d808dbe83dbfbaa" is_pre_election: true
I20251023 20:17:29.360698 20878 raft_consensus.cc:2468] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6503e150f4a04feba7e1331d66c80ac9 in term 0.
I20251023 20:17:29.360940 20952 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "6503e150f4a04feba7e1331d66c80ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9f02e72d02746559133ee5aa66e613c" is_pre_election: true
I20251023 20:17:29.361156 20952 raft_consensus.cc:2468] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6503e150f4a04feba7e1331d66c80ac9 in term 0.
I20251023 20:17:29.361231 20768 leader_election.cc:304] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6503e150f4a04feba7e1331d66c80ac9, 916509046ff043929d808dbe83dbfbaa; no voters:
I20251023 20:17:29.361522 21005 raft_consensus.cc:2804] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:29.361649 21005 raft_consensus.cc:493] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:29.361753 21005 raft_consensus.cc:3060] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:29.364266 21005 raft_consensus.cc:515] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:29.365134 21005 leader_election.cc:290] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [CANDIDATE]: Term 1 election: Requested vote from peers 916509046ff043929d808dbe83dbfbaa (127.15.81.194:40351), e9f02e72d02746559133ee5aa66e613c (127.15.81.195:44181)
I20251023 20:17:29.365392 20878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "6503e150f4a04feba7e1331d66c80ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916509046ff043929d808dbe83dbfbaa"
I20251023 20:17:29.365557 20878 raft_consensus.cc:3060] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:29.366173 20877 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "e9f02e72d02746559133ee5aa66e613c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916509046ff043929d808dbe83dbfbaa" is_pre_election: true
I20251023 20:17:29.366689 20952 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "6503e150f4a04feba7e1331d66c80ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9f02e72d02746559133ee5aa66e613c"
I20251023 20:17:29.366851 20952 raft_consensus.cc:3060] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:29.368146 20878 raft_consensus.cc:2468] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6503e150f4a04feba7e1331d66c80ac9 in term 1.
I20251023 20:17:29.368510 20768 leader_election.cc:304] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6503e150f4a04feba7e1331d66c80ac9, 916509046ff043929d808dbe83dbfbaa; no voters:
I20251023 20:17:29.368659 20803 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "e9f02e72d02746559133ee5aa66e613c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6503e150f4a04feba7e1331d66c80ac9" is_pre_election: true
I20251023 20:17:29.368758 21005 raft_consensus.cc:2804] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:29.369189 21005 raft_consensus.cc:697] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 1 LEADER]: Becoming Leader. State: Replica: 6503e150f4a04feba7e1331d66c80ac9, State: Running, Role: LEADER
I20251023 20:17:29.369266 20952 raft_consensus.cc:2468] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6503e150f4a04feba7e1331d66c80ac9 in term 1.
I20251023 20:17:29.369540 21005 consensus_queue.cc:237] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:29.371328 20918 leader_election.cc:304] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9f02e72d02746559133ee5aa66e613c; no voters: 6503e150f4a04feba7e1331d66c80ac9, 916509046ff043929d808dbe83dbfbaa
I20251023 20:17:29.371556 21006 raft_consensus.cc:2749] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:17:29.372330 20699 catalog_manager.cc:5649] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "6503e150f4a04feba7e1331d66c80ac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:29.422291 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:17:29.428910 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:29.429282 15687 tablet_replica.cc:333] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: stopping tablet replica
I20251023 20:17:29.429453 15687 raft_consensus.cc:2243] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:29.429709 15687 pending_rounds.cc:70] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: Trying to abort 1 pending ops.
I20251023 20:17:29.429774 15687 pending_rounds.cc:77] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082659819675648 op_type: NO_OP noop_request { }
I20251023 20:17:29.429899 15687 raft_consensus.cc:2889] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:29.430011 15687 raft_consensus.cc:2272] T 674c6a7ef701404782d4d90a6323b549 P 6503e150f4a04feba7e1331d66c80ac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:29.433944 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:17:29.744462 20754 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:29.744884 20754 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:29.744961 20754 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:31.310715 21014 raft_consensus.cc:493] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:31.310920 21014 raft_consensus.cc:515] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:31.311705 21014 leader_election.cc:290] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589), 916509046ff043929d808dbe83dbfbaa (127.15.81.194:40351)
I20251023 20:17:31.312475 20878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "e9f02e72d02746559133ee5aa66e613c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916509046ff043929d808dbe83dbfbaa" is_pre_election: true
I20251023 20:17:31.312654 20878 raft_consensus.cc:2468] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9f02e72d02746559133ee5aa66e613c in term 1.
I20251023 20:17:31.313032 20917 leader_election.cc:304] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 916509046ff043929d808dbe83dbfbaa, e9f02e72d02746559133ee5aa66e613c; no voters:
I20251023 20:17:31.313300 21014 raft_consensus.cc:2804] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:17:31.313376 21014 raft_consensus.cc:493] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:31.313426 21014 raft_consensus.cc:3060] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 1 FOLLOWER]: Advancing to term 2
W20251023 20:17:31.313902 20918 leader_election.cc:336] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589): Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111)
I20251023 20:17:31.315429 21014 raft_consensus.cc:515] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:31.315891 21014 leader_election.cc:290] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 2 election: Requested vote from peers 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589), 916509046ff043929d808dbe83dbfbaa (127.15.81.194:40351)
I20251023 20:17:31.316272 20878 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "674c6a7ef701404782d4d90a6323b549" candidate_uuid: "e9f02e72d02746559133ee5aa66e613c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916509046ff043929d808dbe83dbfbaa"
I20251023 20:17:31.316416 20878 raft_consensus.cc:3060] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 1 FOLLOWER]: Advancing to term 2
W20251023 20:17:31.316888 20918 leader_election.cc:336] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589): Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111)
I20251023 20:17:31.318104 20878 raft_consensus.cc:2468] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9f02e72d02746559133ee5aa66e613c in term 2.
I20251023 20:17:31.318427 20917 leader_election.cc:304] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 916509046ff043929d808dbe83dbfbaa, e9f02e72d02746559133ee5aa66e613c; no voters: 6503e150f4a04feba7e1331d66c80ac9
I20251023 20:17:31.318637 21014 raft_consensus.cc:2804] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:17:31.318914 21014 raft_consensus.cc:697] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 2 LEADER]: Becoming Leader. State: Replica: e9f02e72d02746559133ee5aa66e613c, State: Running, Role: LEADER
I20251023 20:17:31.319129 21014 consensus_queue.cc:237] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } }
I20251023 20:17:31.321255 20700 catalog_manager.cc:5649] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c reported cstate change: term changed from 1 to 2, leader changed from 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193) to e9f02e72d02746559133ee5aa66e613c (127.15.81.195). New cstate: current_term: 2 leader_uuid: "e9f02e72d02746559133ee5aa66e613c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6503e150f4a04feba7e1331d66c80ac9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 43589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9f02e72d02746559133ee5aa66e613c" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 44181 } health_report { overall_health: HEALTHY } } }
I20251023 20:17:31.745355 20754 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:31.745720 20754 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:31.745796 20754 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 20:17:31.801652 20918 consensus_peers.cc:597] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c -> Peer 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589): Couldn't send request to peer 6503e150f4a04feba7e1331d66c80ac9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:31.912217 20878 raft_consensus.cc:1275] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 2 FOLLOWER]: Refusing update from remote peer e9f02e72d02746559133ee5aa66e613c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251023 20:17:31.912815 21014 consensus_queue.cc:1048] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [LEADER]: Connected to new peer: Peer: permanent_uuid: "916509046ff043929d808dbe83dbfbaa" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 40351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:17:32.870608 20918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20251023 20:17:33.746129 20754 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:33.746587 20754 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:33.746668 20754 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 20:17:34.472232 20918 consensus_peers.cc:597] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c -> Peer 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589): Couldn't send request to peer 6503e150f4a04feba7e1331d66c80ac9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20251023 20:17:35.747144 20754 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:35.747586 20754 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:35.747663 20754 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 20:17:37.106178 20918 consensus_peers.cc:597] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c -> Peer 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589): Couldn't send request to peer 6503e150f4a04feba7e1331d66c80ac9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251023 20:17:37.748070 20754 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:37.748484 20754 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:37.748560 20754 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251023 20:17:38.106174 20918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20251023 20:17:39.561389 20918 consensus_peers.cc:597] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c -> Peer 6503e150f4a04feba7e1331d66c80ac9 (127.15.81.193:43589): Couldn't send request to peer 6503e150f4a04feba7e1331d66c80ac9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:43589: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251023 20:17:39.749010 20754 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251023 20:17:39.749430 20754 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251023 20:17:39.749517 20754 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251023 20:17:40.490741 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:17:40.510528 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:40.510836 15687 tablet_replica.cc:333] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa: stopping tablet replica
I20251023 20:17:40.511044 15687 raft_consensus.cc:2243] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:40.511230 15687 raft_consensus.cc:2272] T 674c6a7ef701404782d4d90a6323b549 P 916509046ff043929d808dbe83dbfbaa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:40.526188 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:17:40.528733 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:17:40.536765 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:40.537053 15687 tablet_replica.cc:333] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c: stopping tablet replica
I20251023 20:17:40.537213 15687 raft_consensus.cc:2243] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:40.537523 15687 raft_consensus.cc:2272] T 674c6a7ef701404782d4d90a6323b549 P e9f02e72d02746559133ee5aa66e613c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:40.550810 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:17:40.553381 15687 master.cc:561] Master@127.15.81.254:32773 shutting down...
I20251023 20:17:40.560402 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:40.560595 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:40.560703 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P f2329099e07f4ee5ac9ba8b337af2454: stopping tablet replica
I20251023 20:17:40.574368 15687 master.cc:583] Master@127.15.81.254:32773 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor (12889 ms)
[ RUN ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering
I20251023 20:17:40.584591 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:33457
I20251023 20:17:40.585085 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:40.587360 21049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.587422 21050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.587527 21052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:40.587952 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:40.588721 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:40.588786 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:40.588860 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250660588822 us; error 0 us; skew 500 ppm
I20251023 20:17:40.589020 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:40.590086 15687 webserver.cc:492] Webserver started at http://127.15.81.254:36745/ using document root <none> and password file <none>
I20251023 20:17:40.590303 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:40.590387 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:40.590528 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:40.591147 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "81a0b2be187a4234ac49640233315c98"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.593539 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:40.594928 21057 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.595232 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:40.595342 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "81a0b2be187a4234ac49640233315c98"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.595463 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:40.601626 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:40.602085 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:40.613075 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:33457
I20251023 20:17:40.613109 21108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:33457 every 8 connection(s)
I20251023 20:17:40.614171 21109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:40.618013 21109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98: Bootstrap starting.
I20251023 20:17:40.619131 21109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:40.620484 21109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98: No bootstrap required, opened a new log
I20251023 20:17:40.621115 21109 raft_consensus.cc:359] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a0b2be187a4234ac49640233315c98" member_type: VOTER }
I20251023 20:17:40.621254 21109 raft_consensus.cc:385] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:40.621343 21109 raft_consensus.cc:740] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a0b2be187a4234ac49640233315c98, State: Initialized, Role: FOLLOWER
I20251023 20:17:40.621541 21109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a0b2be187a4234ac49640233315c98" member_type: VOTER }
I20251023 20:17:40.621662 21109 raft_consensus.cc:399] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:17:40.621742 21109 raft_consensus.cc:493] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:17:40.621837 21109 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:40.623492 21109 raft_consensus.cc:515] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a0b2be187a4234ac49640233315c98" member_type: VOTER }
I20251023 20:17:40.623677 21109 leader_election.cc:304] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81a0b2be187a4234ac49640233315c98; no voters:
I20251023 20:17:40.624037 21109 leader_election.cc:290] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:17:40.624140 21112 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:40.624596 21112 raft_consensus.cc:697] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 1 LEADER]: Becoming Leader. State: Replica: 81a0b2be187a4234ac49640233315c98, State: Running, Role: LEADER
I20251023 20:17:40.624825 21109 sys_catalog.cc:565] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:17:40.624806 21112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a0b2be187a4234ac49640233315c98" member_type: VOTER }
I20251023 20:17:40.625572 21114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81a0b2be187a4234ac49640233315c98. Latest consensus state: current_term: 1 leader_uuid: "81a0b2be187a4234ac49640233315c98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a0b2be187a4234ac49640233315c98" member_type: VOTER } }
I20251023 20:17:40.625829 21114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:40.626037 21113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81a0b2be187a4234ac49640233315c98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a0b2be187a4234ac49640233315c98" member_type: VOTER } }
I20251023 20:17:40.626214 21113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:40.629316 21117 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:17:40.631481 21117 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:17:40.633893 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:17:40.634799 21117 catalog_manager.cc:1357] Generated new cluster ID: dc93e44b878b4b40b83a719e777ebb4f
I20251023 20:17:40.634887 21117 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:17:40.649147 21117 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:17:40.649832 21117 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:17:40.656267 21117 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98: Generated new TSK 0
I20251023 20:17:40.656474 21117 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:17:40.666594 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:40.668978 21131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.669199 21132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.669670 21134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:40.669942 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:40.670172 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:40.670239 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:40.670301 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250660670281 us; error 0 us; skew 500 ppm
I20251023 20:17:40.670459 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:40.671522 15687 webserver.cc:492] Webserver started at http://127.15.81.193:37935/ using document root <none> and password file <none>
I20251023 20:17:40.671736 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:40.671818 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:40.671964 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:40.672571 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "c3b9f40f341e4b63a82759206f1e882d"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.674953 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:40.676380 21139 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.676748 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:40.676833 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "c3b9f40f341e4b63a82759206f1e882d"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.676970 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:40.707350 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:40.707738 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:40.708142 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:40.708798 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:40.708864 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.708945 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:40.708987 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.719621 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:46417
I20251023 20:17:40.719646 21201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:46417 every 8 connection(s)
I20251023 20:17:40.721154 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:40.723435 21206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.723866 21207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.725050 21209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:40.725311 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:40.725592 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:40.725659 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:40.725688 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250660725685 us; error 0 us; skew 500 ppm
I20251023 20:17:40.725850 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:40.727052 15687 webserver.cc:492] Webserver started at http://127.15.81.194:35375/ using document root <none> and password file <none>
I20251023 20:17:40.727279 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:40.727351 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:40.727466 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:40.727556 21202 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33457
I20251023 20:17:40.727664 21202 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:40.727995 21202 heartbeater.cc:507] Master 127.15.81.254:33457 requested a full tablet report, sending...
I20251023 20:17:40.728171 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.728807 21074 ts_manager.cc:194] Registered new tserver with Master: c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417)
I20251023 20:17:40.730217 21074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40454
I20251023 20:17:40.730928 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20251023 20:17:40.732378 21214 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.732681 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251023 20:17:40.732759 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.732867 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:40.759088 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:40.759703 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:40.760155 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:40.760946 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:40.760998 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.761085 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:40.761122 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.771291 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:43737
I20251023 20:17:40.771319 21277 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:43737 every 8 connection(s)
I20251023 20:17:40.772619 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:40.775239 21281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:40.775830 21282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:40.776543 15687 server_base.cc:1047] running on GCE node
W20251023 20:17:40.777007 21284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:40.777372 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:40.777434 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:40.777500 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250660777491 us; error 0 us; skew 500 ppm
I20251023 20:17:40.777623 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:40.778941 15687 webserver.cc:492] Webserver started at http://127.15.81.195:44497/ using document root <none> and password file <none>
I20251023 20:17:40.779152 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:40.779214 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:40.779237 21278 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33457
I20251023 20:17:40.779331 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:40.779340 21278 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:40.779621 21278 heartbeater.cc:507] Master 127.15.81.254:33457 requested a full tablet report, sending...
I20251023 20:17:40.779925 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "53ede9ebe3094cf99e1a5286a164319d"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.780340 21074 ts_manager.cc:194] Registered new tserver with Master: 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:40.781042 21074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40466
I20251023 20:17:40.782372 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:40.783620 21289 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.783880 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:40.783953 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "53ede9ebe3094cf99e1a5286a164319d"
format_stamp: "Formatted at 2025-10-23 20:17:40 on dist-test-slave-jr94"
I20251023 20:17:40.784036 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:40.799209 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:40.799628 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:40.800055 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:40.800719 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:40.800776 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.800858 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:40.800912 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:40.812528 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:33329
I20251023 20:17:40.812553 21351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:33329 every 8 connection(s)
I20251023 20:17:40.818040 21352 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33457
I20251023 20:17:40.818145 21352 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:40.818415 21352 heartbeater.cc:507] Master 127.15.81.254:33457 requested a full tablet report, sending...
I20251023 20:17:40.819049 21074 ts_manager.cc:194] Registered new tserver with Master: 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
I20251023 20:17:40.819718 21074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40470
I20251023 20:17:40.819934 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006684032s
I20251023 20:17:41.731493 21202 heartbeater.cc:499] Master 127.15.81.254:33457 was elected leader, sending a full tablet report...
I20251023 20:17:41.782114 21278 heartbeater.cc:499] Master 127.15.81.254:33457 was elected leader, sending a full tablet report...
I20251023 20:17:41.820881 21352 heartbeater.cc:499] Master 127.15.81.254:33457 was elected leader, sending a full tablet report...
I20251023 20:17:41.847278 15687 test_util.cc:276] Using random seed: 537874639
I20251023 20:17:41.857746 21074 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40486:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:17:41.858414 21074 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:17:41.879552 21317 tablet_service.cc:1505] Processing CreateTablet for tablet 4cf4d2ea7b734f6db1f467510835c352 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:17:41.879635 21315 tablet_service.cc:1505] Processing CreateTablet for tablet e2997cd6487c4020965aa97ceb29b288 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:17:41.880067 21317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf4d2ea7b734f6db1f467510835c352. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.880271 21315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2997cd6487c4020965aa97ceb29b288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.882448 21242 tablet_service.cc:1505] Processing CreateTablet for tablet 4cf4d2ea7b734f6db1f467510835c352 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:17:41.882532 21241 tablet_service.cc:1505] Processing CreateTablet for tablet 0c966dadbdc14249b3b1d23723a0f469 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:17:41.882746 21242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf4d2ea7b734f6db1f467510835c352. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.882872 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c966dadbdc14249b3b1d23723a0f469. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.885140 21316 tablet_service.cc:1505] Processing CreateTablet for tablet 0c966dadbdc14249b3b1d23723a0f469 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:17:41.885478 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c966dadbdc14249b3b1d23723a0f469. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.886529 21372 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d: Bootstrap starting.
I20251023 20:17:41.888170 21373 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Bootstrap starting.
I20251023 20:17:41.888435 21372 tablet_bootstrap.cc:654] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.889220 21314 tablet_service.cc:1505] Processing CreateTablet for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:17:41.889592 21314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.889997 21373 tablet_bootstrap.cc:654] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.891952 21373 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: No bootstrap required, opened a new log
I20251023 20:17:41.892100 21373 ts_tablet_manager.cc:1403] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:17:41.892904 21373 raft_consensus.cc:359] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.893100 21373 raft_consensus.cc:385] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.893191 21373 raft_consensus.cc:740] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.893307 21240 tablet_service.cc:1505] Processing CreateTablet for tablet e2997cd6487c4020965aa97ceb29b288 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:17:41.893478 21373 consensus_queue.cc:260] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.893671 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2997cd6487c4020965aa97ceb29b288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.897274 21239 tablet_service.cc:1505] Processing CreateTablet for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:17:41.897722 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.899917 21373 ts_tablet_manager.cc:1434] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20251023 20:17:41.904976 21373 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Bootstrap starting.
I20251023 20:17:41.908296 21373 tablet_bootstrap.cc:654] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.915177 21167 tablet_service.cc:1505] Processing CreateTablet for tablet 4cf4d2ea7b734f6db1f467510835c352 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:17:41.915639 21164 tablet_service.cc:1505] Processing CreateTablet for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:17:41.915736 21167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf4d2ea7b734f6db1f467510835c352. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.915974 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.920341 21165 tablet_service.cc:1505] Processing CreateTablet for tablet e2997cd6487c4020965aa97ceb29b288 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:17:41.920621 21372 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d: No bootstrap required, opened a new log
I20251023 20:17:41.920670 21165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2997cd6487c4020965aa97ceb29b288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.920734 21372 ts_tablet_manager.cc:1403] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.015s
I20251023 20:17:41.915263 21166 tablet_service.cc:1505] Processing CreateTablet for tablet 0c966dadbdc14249b3b1d23723a0f469 (DEFAULT_TABLE table=test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:17:41.921361 21166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c966dadbdc14249b3b1d23723a0f469. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:41.921593 21372 raft_consensus.cc:359] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.921744 21372 raft_consensus.cc:385] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.921792 21372 raft_consensus.cc:740] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.922012 21372 consensus_queue.cc:260] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.924714 21373 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: No bootstrap required, opened a new log
I20251023 20:17:41.924824 21373 ts_tablet_manager.cc:1403] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.008s
I20251023 20:17:41.925586 21373 raft_consensus.cc:359] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.925796 21373 raft_consensus.cc:385] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.925937 21373 raft_consensus.cc:740] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.926187 21373 consensus_queue.cc:260] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.927841 21373 ts_tablet_manager.cc:1434] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251023 20:17:41.928157 21373 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Bootstrap starting.
I20251023 20:17:41.930729 21373 tablet_bootstrap.cc:654] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.930944 21372 ts_tablet_manager.cc:1434] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d: Time spent starting tablet: real 0.010s user 0.000s sys 0.005s
I20251023 20:17:41.931135 21378 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d: Bootstrap starting.
I20251023 20:17:41.931416 21372 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: Bootstrap starting.
I20251023 20:17:41.933252 21378 tablet_bootstrap.cc:654] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.933755 21372 tablet_bootstrap.cc:654] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.933398 21379 raft_consensus.cc:493] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:41.933962 21379 raft_consensus.cc:515] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.935173 21379 leader_election.cc:290] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:41.936347 21372 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: No bootstrap required, opened a new log
I20251023 20:17:41.936440 21373 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: No bootstrap required, opened a new log
I20251023 20:17:41.936496 21372 ts_tablet_manager.cc:1403] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251023 20:17:41.936578 21373 ts_tablet_manager.cc:1403] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20251023 20:17:41.937299 21372 raft_consensus.cc:359] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.937505 21372 raft_consensus.cc:385] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.937595 21372 raft_consensus.cc:740] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.937570 21373 raft_consensus.cc:359] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.937872 21373 raft_consensus.cc:385] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.938004 21373 raft_consensus.cc:740] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.938287 21373 consensus_queue.cc:260] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.937839 21372 consensus_queue.cc:260] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.938941 21372 ts_tablet_manager.cc:1434] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:17:41.940582 21372 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: Bootstrap starting.
I20251023 20:17:41.940744 21378 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d: No bootstrap required, opened a new log
I20251023 20:17:41.940901 21378 ts_tablet_manager.cc:1403] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20251023 20:17:41.941675 21378 raft_consensus.cc:359] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.941884 21378 raft_consensus.cc:385] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.941937 21378 raft_consensus.cc:740] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b9f40f341e4b63a82759206f1e882d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.942189 21378 consensus_queue.cc:260] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.942366 21372 tablet_bootstrap.cc:654] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.943974 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d" is_pre_election: true
I20251023 20:17:41.944360 21373 ts_tablet_manager.cc:1434] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20251023 20:17:41.944597 21373 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Bootstrap starting.
I20251023 20:17:41.946134 21373 tablet_bootstrap.cc:654] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Neither blocks nor log segments found. Creating new log.
W20251023 20:17:41.946177 21293 leader_election.cc:343] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:17:41.947105 21378 ts_tablet_manager.cc:1434] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20251023 20:17:41.948195 21372 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: No bootstrap required, opened a new log
I20251023 20:17:41.948391 21378 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d: Bootstrap starting.
I20251023 20:17:41.948390 21372 ts_tablet_manager.cc:1403] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251023 20:17:41.949213 21372 raft_consensus.cc:359] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.949345 21372 raft_consensus.cc:385] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.950161 21378 tablet_bootstrap.cc:654] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.950304 21373 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: No bootstrap required, opened a new log
I20251023 20:17:41.950428 21373 ts_tablet_manager.cc:1403] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251023 20:17:41.950598 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" is_pre_election: true
W20251023 20:17:41.951280 21293 leader_election.cc:343] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737): Illegal state: must be running to vote when last-logged opid is not known
I20251023 20:17:41.951289 21372 raft_consensus.cc:740] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.951231 21373 raft_consensus.cc:359] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.951435 21373 raft_consensus.cc:385] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.951416 21293 leader_election.cc:304] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d; no voters: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, c3b9f40f341e4b63a82759206f1e882d
I20251023 20:17:41.951553 21372 consensus_queue.cc:260] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.951822 21379 raft_consensus.cc:2749] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 20:17:41.952101 21378 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d: No bootstrap required, opened a new log
I20251023 20:17:41.952227 21378 ts_tablet_manager.cc:1403] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:17:41.952332 21372 ts_tablet_manager.cc:1434] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:17:41.952522 21373 raft_consensus.cc:740] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.952744 21372 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: Bootstrap starting.
I20251023 20:17:41.952756 21373 consensus_queue.cc:260] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.952963 21378 raft_consensus.cc:359] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.953148 21378 raft_consensus.cc:385] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.953259 21378 raft_consensus.cc:740] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b9f40f341e4b63a82759206f1e882d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.953473 21378 consensus_queue.cc:260] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.953936 21378 ts_tablet_manager.cc:1434] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:17:41.953954 21373 ts_tablet_manager.cc:1434] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251023 20:17:41.954198 21378 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d: Bootstrap starting.
I20251023 20:17:41.955811 21378 tablet_bootstrap.cc:654] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.956607 21372 tablet_bootstrap.cc:654] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.958213 21378 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d: No bootstrap required, opened a new log
I20251023 20:17:41.958339 21378 ts_tablet_manager.cc:1403] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 20:17:41.958927 21372 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: No bootstrap required, opened a new log
I20251023 20:17:41.959048 21372 ts_tablet_manager.cc:1403] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20251023 20:17:41.958988 21378 raft_consensus.cc:359] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.959156 21378 raft_consensus.cc:385] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.959230 21378 raft_consensus.cc:740] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b9f40f341e4b63a82759206f1e882d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.959398 21378 consensus_queue.cc:260] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.959681 21372 raft_consensus.cc:359] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.959821 21372 raft_consensus.cc:385] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.959856 21378 ts_tablet_manager.cc:1434] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:41.959890 21372 raft_consensus.cc:740] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.960129 21378 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: Bootstrap starting.
I20251023 20:17:41.960098 21372 consensus_queue.cc:260] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.960536 21372 ts_tablet_manager.cc:1434] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:41.961885 21378 tablet_bootstrap.cc:654] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:41.963507 21378 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: No bootstrap required, opened a new log
I20251023 20:17:41.963624 21378 ts_tablet_manager.cc:1403] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:17:41.964191 21378 raft_consensus.cc:359] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.964310 21378 raft_consensus.cc:385] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:41.964367 21378 raft_consensus.cc:740] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b9f40f341e4b63a82759206f1e882d, State: Initialized, Role: FOLLOWER
I20251023 20:17:41.964497 21378 consensus_queue.cc:260] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.964810 21378 ts_tablet_manager.cc:1434] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:41.966104 21379 raft_consensus.cc:493] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:41.966230 21379 raft_consensus.cc:515] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.966734 21379 leader_election.cc:290] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737), c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417)
I20251023 20:17:41.966943 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4cf4d2ea7b734f6db1f467510835c352" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" is_pre_election: true
I20251023 20:17:41.967038 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4cf4d2ea7b734f6db1f467510835c352" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d" is_pre_election: true
I20251023 20:17:41.967132 21252 raft_consensus.cc:2468] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 0.
I20251023 20:17:41.967196 21177 raft_consensus.cc:2468] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 0.
I20251023 20:17:41.967459 21293 leader_election.cc:304] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, 9ac7ad9ef0bc478d95c036f4fb6ea7ff; no voters:
I20251023 20:17:41.967670 21379 raft_consensus.cc:2804] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:41.967751 21379 raft_consensus.cc:493] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:41.967831 21379 raft_consensus.cc:3060] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:41.969767 21379 raft_consensus.cc:515] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.970211 21379 leader_election.cc:290] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 election: Requested vote from peers 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737), c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417)
I20251023 20:17:41.970567 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4cf4d2ea7b734f6db1f467510835c352" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff"
I20251023 20:17:41.970571 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4cf4d2ea7b734f6db1f467510835c352" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d"
I20251023 20:17:41.970712 21252 raft_consensus.cc:3060] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:41.970750 21177 raft_consensus.cc:3060] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:41.972618 21177 raft_consensus.cc:2468] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 1.
I20251023 20:17:41.972791 21252 raft_consensus.cc:2468] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 1.
I20251023 20:17:41.972983 21293 leader_election.cc:304] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, c3b9f40f341e4b63a82759206f1e882d; no voters:
I20251023 20:17:41.973238 21379 raft_consensus.cc:2804] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:41.973356 21379 raft_consensus.cc:697] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Becoming Leader. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Running, Role: LEADER
I20251023 20:17:41.973560 21379 consensus_queue.cc:237] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } }
I20251023 20:17:41.975521 21074 catalog_manager.cc:5649] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195). New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:41.994349 21379 raft_consensus.cc:493] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:41.994496 21379 raft_consensus.cc:515] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.994933 21379 leader_election.cc:290] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:41.995262 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c966dadbdc14249b3b1d23723a0f469" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d" is_pre_election: true
I20251023 20:17:41.995333 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c966dadbdc14249b3b1d23723a0f469" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" is_pre_election: true
I20251023 20:17:41.995431 21177 raft_consensus.cc:2468] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 0.
I20251023 20:17:41.995522 21252 raft_consensus.cc:2468] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 0.
I20251023 20:17:41.995756 21293 leader_election.cc:304] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, c3b9f40f341e4b63a82759206f1e882d; no voters:
I20251023 20:17:41.995957 21379 raft_consensus.cc:2804] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:41.996047 21379 raft_consensus.cc:493] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:41.996140 21379 raft_consensus.cc:3060] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:41.997896 21379 raft_consensus.cc:515] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:41.998291 21379 leader_election.cc:290] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 election: Requested vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:41.998570 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c966dadbdc14249b3b1d23723a0f469" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d"
I20251023 20:17:41.998625 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c966dadbdc14249b3b1d23723a0f469" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff"
I20251023 20:17:41.998754 21177 raft_consensus.cc:3060] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:41.998777 21252 raft_consensus.cc:3060] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.001295 21252 raft_consensus.cc:2468] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 1.
I20251023 20:17:42.001295 21177 raft_consensus.cc:2468] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 1.
I20251023 20:17:42.001686 21293 leader_election.cc:304] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, 9ac7ad9ef0bc478d95c036f4fb6ea7ff; no voters:
I20251023 20:17:42.001929 21379 raft_consensus.cc:2804] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:42.002085 21379 raft_consensus.cc:697] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Becoming Leader. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Running, Role: LEADER
I20251023 20:17:42.002352 21379 consensus_queue.cc:237] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.004259 21074 catalog_manager.cc:5649] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195). New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } }
I20251023 20:17:42.219501 21385 raft_consensus.cc:493] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:42.219740 21385 raft_consensus.cc:515] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.220579 21385 leader_election.cc:290] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737), 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
I20251023 20:17:42.221267 21379 raft_consensus.cc:493] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:42.221431 21379 raft_consensus.cc:515] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.222119 21379 leader_election.cc:290] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:42.222836 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fba655f10914ec5a3bbdbd7dcf8ab53" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" is_pre_election: true
I20251023 20:17:42.222800 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fba655f10914ec5a3bbdbd7dcf8ab53" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d" is_pre_election: true
I20251023 20:17:42.223155 21252 raft_consensus.cc:2468] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 0.
I20251023 20:17:42.223192 21177 raft_consensus.cc:2468] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 0.
I20251023 20:17:42.223552 21293 leader_election.cc:304] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, 9ac7ad9ef0bc478d95c036f4fb6ea7ff; no voters:
I20251023 20:17:42.223843 21379 raft_consensus.cc:2804] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:42.223949 21379 raft_consensus.cc:493] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:42.224076 21379 raft_consensus.cc:3060] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.227164 21379 raft_consensus.cc:515] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.227784 21379 leader_election.cc:290] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 election: Requested vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:42.227892 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "c3b9f40f341e4b63a82759206f1e882d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" is_pre_election: true
I20251023 20:17:42.228073 21252 raft_consensus.cc:2468] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b9f40f341e4b63a82759206f1e882d in term 0.
I20251023 20:17:42.228161 21251 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fba655f10914ec5a3bbdbd7dcf8ab53" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff"
I20251023 20:17:42.228386 21251 raft_consensus.cc:3060] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.228361 21327 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "c3b9f40f341e4b63a82759206f1e882d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ede9ebe3094cf99e1a5286a164319d" is_pre_election: true
I20251023 20:17:42.228551 21327 raft_consensus.cc:2468] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b9f40f341e4b63a82759206f1e882d in term 0.
I20251023 20:17:42.228809 21143 leader_election.cc:304] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, c3b9f40f341e4b63a82759206f1e882d; no voters:
I20251023 20:17:42.228559 21177 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fba655f10914ec5a3bbdbd7dcf8ab53" candidate_uuid: "53ede9ebe3094cf99e1a5286a164319d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b9f40f341e4b63a82759206f1e882d"
I20251023 20:17:42.229188 21385 raft_consensus.cc:2804] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:42.229209 21177 raft_consensus.cc:3060] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.229271 21385 raft_consensus.cc:493] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:42.229331 21385 raft_consensus.cc:3060] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.231387 21385 raft_consensus.cc:515] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.231593 21251 raft_consensus.cc:2468] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 1.
I20251023 20:17:42.231909 21385 leader_election.cc:290] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [CANDIDATE]: Term 1 election: Requested vote from peers 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737), 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
I20251023 20:17:42.232062 21177 raft_consensus.cc:2468] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ede9ebe3094cf99e1a5286a164319d in term 1.
I20251023 20:17:42.232117 21293 leader_election.cc:304] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, 9ac7ad9ef0bc478d95c036f4fb6ea7ff; no voters:
I20251023 20:17:42.232275 21327 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "c3b9f40f341e4b63a82759206f1e882d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ede9ebe3094cf99e1a5286a164319d"
I20251023 20:17:42.232430 21379 raft_consensus.cc:2804] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:42.232491 21327 raft_consensus.cc:3060] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.232579 21379 raft_consensus.cc:697] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Becoming Leader. State: Replica: 53ede9ebe3094cf99e1a5286a164319d, State: Running, Role: LEADER
I20251023 20:17:42.232765 21252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "c3b9f40f341e4b63a82759206f1e882d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff"
I20251023 20:17:42.232991 21252 raft_consensus.cc:3060] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:42.232905 21379 consensus_queue.cc:237] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.235311 21327 raft_consensus.cc:2468] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b9f40f341e4b63a82759206f1e882d in term 1.
I20251023 20:17:42.235586 21252 raft_consensus.cc:2468] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b9f40f341e4b63a82759206f1e882d in term 1.
I20251023 20:17:42.235777 21141 leader_election.cc:304] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, c3b9f40f341e4b63a82759206f1e882d; no voters:
I20251023 20:17:42.236186 21385 raft_consensus.cc:2804] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:42.236685 21385 raft_consensus.cc:697] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 1 LEADER]: Becoming Leader. State: Replica: c3b9f40f341e4b63a82759206f1e882d, State: Running, Role: LEADER
I20251023 20:17:42.236975 21385 consensus_queue.cc:237] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:42.237327 21074 catalog_manager.cc:5649] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195). New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } }
I20251023 20:17:42.239279 21072 catalog_manager.cc:5649] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d reported cstate change: term changed from 0 to 1, leader changed from <none> to c3b9f40f341e4b63a82759206f1e882d (127.15.81.193). New cstate: current_term: 1 leader_uuid: "c3b9f40f341e4b63a82759206f1e882d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:42.317252 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:17:42.324345 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:42.324733 15687 tablet_replica.cc:333] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: stopping tablet replica
I20251023 20:17:42.324952 15687 raft_consensus.cc:2243] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:42.325209 15687 pending_rounds.cc:70] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: Trying to abort 1 pending ops.
I20251023 20:17:42.325273 15687 pending_rounds.cc:77] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082712524939264 op_type: NO_OP noop_request { }
I20251023 20:17:42.325394 15687 raft_consensus.cc:2889] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:42.325502 15687 raft_consensus.cc:2272] T e2997cd6487c4020965aa97ceb29b288 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:42.326334 15687 tablet_replica.cc:333] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d: stopping tablet replica
I20251023 20:17:42.326475 15687 raft_consensus.cc:2243] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:42.326622 15687 raft_consensus.cc:2272] T 4cf4d2ea7b734f6db1f467510835c352 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:42.327100 15687 tablet_replica.cc:333] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d: stopping tablet replica
I20251023 20:17:42.327230 15687 raft_consensus.cc:2243] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:42.327359 15687 raft_consensus.cc:2272] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:42.327805 15687 tablet_replica.cc:333] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d: stopping tablet replica
I20251023 20:17:42.327934 15687 raft_consensus.cc:2243] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:42.328058 15687 raft_consensus.cc:2272] T 0c966dadbdc14249b3b1d23723a0f469 P c3b9f40f341e4b63a82759206f1e882d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:42.342677 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:17:42.356197 21379 consensus_queue.cc:1048] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:17:42.360822 21293 consensus_peers.cc:597] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 20:17:42.587333 21293 consensus_peers.cc:597] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:42.614382 21379 consensus_queue.cc:1048] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:17:42.716078 21293 consensus_peers.cc:597] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:42.801952 21389 consensus_queue.cc:1048] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:43.186388 21389 consensus_queue.cc:579] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.184s)
W20251023 20:17:43.187252 21293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20251023 20:17:43.189060 21074 catalog_manager.cc:5327] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0c966dadbdc14249b3b1d23723a0f469 with cas_config_opid_index -1: no extra replica candidate found for tablet 0c966dadbdc14249b3b1d23723a0f469 (table test-workload [id=6bba7c664821493aa5669b5b0c52b1ce]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20251023 20:17:43.251039 21379 consensus_queue.cc:579] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.018s)
I20251023 20:17:43.275516 21379 consensus_queue.cc:579] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.301s)
I20251023 20:17:43.346491 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:43.349220 21409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:43.349611 21408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:43.349804 21411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:43.350271 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:43.363299 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:43.363379 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:43.363405 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250663363400 us; error 0 us; skew 500 ppm
I20251023 20:17:43.363512 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:43.364687 15687 webserver.cc:492] Webserver started at http://127.15.81.196:33663/ using document root <none> and password file <none>
I20251023 20:17:43.364917 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:43.364979 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:43.365077 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:43.365665 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "efe4dc319f25495580e38f8a19b2b7d4"
format_stamp: "Formatted at 2025-10-23 20:17:43 on dist-test-slave-jr94"
I20251023 20:17:43.367908 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 20:17:43.369596 21416 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:43.369944 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251023 20:17:43.370026 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "efe4dc319f25495580e38f8a19b2b7d4"
format_stamp: "Formatted at 2025-10-23 20:17:43 on dist-test-slave-jr94"
I20251023 20:17:43.370105 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:43.380234 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:43.380628 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:43.381161 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:43.382009 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:43.382059 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:43.382112 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:43.382150 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:43.392369 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:42139
I20251023 20:17:43.392381 21478 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:42139 every 8 connection(s)
I20251023 20:17:43.398890 21479 heartbeater.cc:344] Connected to a master server at 127.15.81.254:33457
I20251023 20:17:43.398996 21479 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:43.399291 21479 heartbeater.cc:507] Master 127.15.81.254:33457 requested a full tablet report, sending...
I20251023 20:17:43.399963 21074 ts_manager.cc:194] Registered new tserver with Master: efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196:42139)
I20251023 20:17:43.400591 21074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40492
I20251023 20:17:43.419461 21327 consensus_queue.cc:237] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:43.421764 21252 raft_consensus.cc:1275] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:17:43.422561 21406 consensus_queue.cc:1048] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 20:17:43.423106 21293 consensus_peers.cc:597] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:43.424723 21379 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.425794 21252 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.428309 21059 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
W20251023 20:17:43.429047 21292 consensus_peers.cc:597] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d -> Peer efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196:42139): Couldn't send request to peer efe4dc319f25495580e38f8a19b2b7d4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8fba655f10914ec5a3bbdbd7dcf8ab53. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:43.429935 21074 catalog_manager.cc:5649] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:43.477456 21327 consensus_queue.cc:237] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
W20251023 20:17:43.479842 21292 consensus_peers.cc:597] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d -> Peer efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196:42139): Couldn't send request to peer efe4dc319f25495580e38f8a19b2b7d4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cf4d2ea7b734f6db1f467510835c352. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:43.479887 21252 raft_consensus.cc:1275] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:17:43.480424 21379 consensus_queue.cc:1048] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 20:17:43.480582 21293 consensus_peers.cc:597] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:43.482482 21406 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.482980 21252 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.485060 21059 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4cf4d2ea7b734f6db1f467510835c352 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
I20251023 20:17:43.486310 21072 catalog_manager.cc:5649] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff reported cstate change: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.523608 21327 consensus_queue.cc:237] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
W20251023 20:17:43.525501 21292 consensus_peers.cc:597] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d -> Peer efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196:42139): Couldn't send request to peer efe4dc319f25495580e38f8a19b2b7d4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c966dadbdc14249b3b1d23723a0f469. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:43.525808 21252 raft_consensus.cc:1275] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 20:17:43.526177 21293 consensus_peers.cc:597] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:43.526197 21406 consensus_queue.cc:1048] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:17:43.527806 21379 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.528311 21252 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:43.530339 21059 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0c966dadbdc14249b3b1d23723a0f469 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 5)
I20251023 20:17:43.531451 21074 catalog_manager.cc:5649] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:17:43.634558 21122 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver c3b9f40f341e4b63a82759206f1e882d not available for placement
I20251023 20:17:43.962119 21493 ts_tablet_manager.cc:933] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: Initiating tablet copy from peer 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
I20251023 20:17:43.962620 21493 tablet_copy_client.cc:323] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:33329
I20251023 20:17:43.967491 21337 tablet_copy_service.cc:140] P 53ede9ebe3094cf99e1a5286a164319d: Received BeginTabletCopySession request for tablet 0c966dadbdc14249b3b1d23723a0f469 from peer efe4dc319f25495580e38f8a19b2b7d4 ({username='slave'} at 127.0.0.1:58990)
I20251023 20:17:43.967612 21337 tablet_copy_service.cc:161] P 53ede9ebe3094cf99e1a5286a164319d: Beginning new tablet copy session on tablet 0c966dadbdc14249b3b1d23723a0f469 from peer efe4dc319f25495580e38f8a19b2b7d4 at {username='slave'} at 127.0.0.1:58990: session id = efe4dc319f25495580e38f8a19b2b7d4-0c966dadbdc14249b3b1d23723a0f469
I20251023 20:17:43.969516 21337 tablet_copy_source_session.cc:215] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:17:43.970422 21493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c966dadbdc14249b3b1d23723a0f469. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:43.974366 21493 tablet_copy_client.cc:806] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 0 data blocks...
I20251023 20:17:43.974591 21493 tablet_copy_client.cc:670] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 1 WAL segments...
I20251023 20:17:43.975793 21493 tablet_copy_client.cc:538] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:43.977771 21493 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap starting.
I20251023 20:17:43.981724 21493 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:43.981988 21493 tablet_bootstrap.cc:492] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap complete.
I20251023 20:17:43.982160 21493 ts_tablet_manager.cc:1403] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 20:17:43.982623 21493 raft_consensus.cc:359] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:43.982756 21493 raft_consensus.cc:740] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: efe4dc319f25495580e38f8a19b2b7d4, State: Initialized, Role: LEARNER
I20251023 20:17:43.982875 21493 consensus_queue.cc:260] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:43.983419 21493 ts_tablet_manager.cc:1434] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 20:17:43.983453 21479 heartbeater.cc:499] Master 127.15.81.254:33457 was elected leader, sending a full tablet report...
I20251023 20:17:43.983961 21337 tablet_copy_service.cc:342] P 53ede9ebe3094cf99e1a5286a164319d: Request end of tablet copy session efe4dc319f25495580e38f8a19b2b7d4-0c966dadbdc14249b3b1d23723a0f469 received from {username='slave'} at 127.0.0.1:58990
I20251023 20:17:43.984063 21337 tablet_copy_service.cc:434] P 53ede9ebe3094cf99e1a5286a164319d: ending tablet copy session efe4dc319f25495580e38f8a19b2b7d4-0c966dadbdc14249b3b1d23723a0f469 on tablet 0c966dadbdc14249b3b1d23723a0f469 with peer efe4dc319f25495580e38f8a19b2b7d4
I20251023 20:17:44.007889 21485 raft_consensus.cc:493] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:44.008013 21485 raft_consensus.cc:515] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:44.008682 21485 leader_election.cc:290] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
W20251023 20:17:44.010722 21218 leader_election.cc:336] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
I20251023 20:17:44.013895 21327 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ede9ebe3094cf99e1a5286a164319d" is_pre_election: true
I20251023 20:17:44.014032 21327 raft_consensus.cc:2468] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ac7ad9ef0bc478d95c036f4fb6ea7ff in term 1.
I20251023 20:17:44.014353 21216 leader_election.cc:304] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, 9ac7ad9ef0bc478d95c036f4fb6ea7ff; no voters: c3b9f40f341e4b63a82759206f1e882d
I20251023 20:17:44.014544 21485 raft_consensus.cc:2804] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:17:44.014608 21485 raft_consensus.cc:493] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:44.014650 21485 raft_consensus.cc:3060] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:44.016321 21485 raft_consensus.cc:515] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:44.016783 21485 leader_election.cc:290] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [CANDIDATE]: Term 2 election: Requested vote from peers c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417), 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
I20251023 20:17:44.017264 21327 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e2997cd6487c4020965aa97ceb29b288" candidate_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ede9ebe3094cf99e1a5286a164319d"
I20251023 20:17:44.017432 21327 raft_consensus.cc:3060] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:17:44.017668 21493 ts_tablet_manager.cc:933] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: Initiating tablet copy from peer 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
W20251023 20:17:44.017894 21218 leader_election.cc:336] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
I20251023 20:17:44.018361 21493 tablet_copy_client.cc:323] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:33329
I20251023 20:17:44.018899 21337 tablet_copy_service.cc:140] P 53ede9ebe3094cf99e1a5286a164319d: Received BeginTabletCopySession request for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 from peer efe4dc319f25495580e38f8a19b2b7d4 ({username='slave'} at 127.0.0.1:58990)
I20251023 20:17:44.018990 21337 tablet_copy_service.cc:161] P 53ede9ebe3094cf99e1a5286a164319d: Beginning new tablet copy session on tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 from peer efe4dc319f25495580e38f8a19b2b7d4 at {username='slave'} at 127.0.0.1:58990: session id = efe4dc319f25495580e38f8a19b2b7d4-8fba655f10914ec5a3bbdbd7dcf8ab53
I20251023 20:17:44.019656 21327 raft_consensus.cc:2468] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ac7ad9ef0bc478d95c036f4fb6ea7ff in term 2.
I20251023 20:17:44.019954 21216 leader_election.cc:304] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53ede9ebe3094cf99e1a5286a164319d, 9ac7ad9ef0bc478d95c036f4fb6ea7ff; no voters: c3b9f40f341e4b63a82759206f1e882d
I20251023 20:17:44.020211 21485 raft_consensus.cc:2804] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:17:44.020552 21485 raft_consensus.cc:697] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 LEADER]: Becoming Leader. State: Replica: 9ac7ad9ef0bc478d95c036f4fb6ea7ff, State: Running, Role: LEADER
I20251023 20:17:44.020781 21485 consensus_queue.cc:237] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } }
I20251023 20:17:44.021409 21337 tablet_copy_source_session.cc:215] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:17:44.022477 21493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:44.023101 21072 catalog_manager.cc:5649] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff reported cstate change: term changed from 1 to 2, leader changed from c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) to 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194). New cstate: current_term: 2 leader_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:44.026793 21493 tablet_copy_client.cc:806] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 0 data blocks...
I20251023 20:17:44.026986 21493 tablet_copy_client.cc:670] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 1 WAL segments...
I20251023 20:17:44.028059 21493 tablet_copy_client.cc:538] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:44.030023 21493 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap starting.
I20251023 20:17:44.033857 21493 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:44.034106 21493 tablet_bootstrap.cc:492] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap complete.
I20251023 20:17:44.034282 21493 ts_tablet_manager.cc:1403] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20251023 20:17:44.034752 21493 raft_consensus.cc:359] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:44.034896 21493 raft_consensus.cc:740] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: efe4dc319f25495580e38f8a19b2b7d4, State: Initialized, Role: LEARNER
I20251023 20:17:44.035055 21493 consensus_queue.cc:260] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:44.035372 21493 ts_tablet_manager.cc:1434] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:44.035912 21337 tablet_copy_service.cc:342] P 53ede9ebe3094cf99e1a5286a164319d: Request end of tablet copy session efe4dc319f25495580e38f8a19b2b7d4-8fba655f10914ec5a3bbdbd7dcf8ab53 received from {username='slave'} at 127.0.0.1:58990
I20251023 20:17:44.036002 21337 tablet_copy_service.cc:434] P 53ede9ebe3094cf99e1a5286a164319d: ending tablet copy session efe4dc319f25495580e38f8a19b2b7d4-8fba655f10914ec5a3bbdbd7dcf8ab53 on tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 with peer efe4dc319f25495580e38f8a19b2b7d4
I20251023 20:17:44.086658 21493 ts_tablet_manager.cc:933] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: Initiating tablet copy from peer 53ede9ebe3094cf99e1a5286a164319d (127.15.81.195:33329)
I20251023 20:17:44.087134 21493 tablet_copy_client.cc:323] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:33329
I20251023 20:17:44.087678 21337 tablet_copy_service.cc:140] P 53ede9ebe3094cf99e1a5286a164319d: Received BeginTabletCopySession request for tablet 4cf4d2ea7b734f6db1f467510835c352 from peer efe4dc319f25495580e38f8a19b2b7d4 ({username='slave'} at 127.0.0.1:58990)
I20251023 20:17:44.087781 21337 tablet_copy_service.cc:161] P 53ede9ebe3094cf99e1a5286a164319d: Beginning new tablet copy session on tablet 4cf4d2ea7b734f6db1f467510835c352 from peer efe4dc319f25495580e38f8a19b2b7d4 at {username='slave'} at 127.0.0.1:58990: session id = efe4dc319f25495580e38f8a19b2b7d4-4cf4d2ea7b734f6db1f467510835c352
I20251023 20:17:44.089567 21337 tablet_copy_source_session.cc:215] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:17:44.090435 21493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf4d2ea7b734f6db1f467510835c352. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:44.094456 21493 tablet_copy_client.cc:806] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 0 data blocks...
I20251023 20:17:44.094637 21493 tablet_copy_client.cc:670] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 1 WAL segments...
I20251023 20:17:44.095737 21493 tablet_copy_client.cc:538] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:44.097967 21493 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap starting.
I20251023 20:17:44.101879 21493 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:44.102166 21493 tablet_bootstrap.cc:492] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap complete.
I20251023 20:17:44.102393 21493 ts_tablet_manager.cc:1403] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20251023 20:17:44.102892 21493 raft_consensus.cc:359] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:44.103024 21493 raft_consensus.cc:740] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: efe4dc319f25495580e38f8a19b2b7d4, State: Initialized, Role: LEARNER
I20251023 20:17:44.103200 21493 consensus_queue.cc:260] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:44.103577 21493 ts_tablet_manager.cc:1434] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:44.104138 21337 tablet_copy_service.cc:342] P 53ede9ebe3094cf99e1a5286a164319d: Request end of tablet copy session efe4dc319f25495580e38f8a19b2b7d4-4cf4d2ea7b734f6db1f467510835c352 received from {username='slave'} at 127.0.0.1:58990
I20251023 20:17:44.104230 21337 tablet_copy_service.cc:434] P 53ede9ebe3094cf99e1a5286a164319d: ending tablet copy session efe4dc319f25495580e38f8a19b2b7d4-4cf4d2ea7b734f6db1f467510835c352 on tablet 4cf4d2ea7b734f6db1f467510835c352 with peer efe4dc319f25495580e38f8a19b2b7d4
I20251023 20:17:44.420418 21406 consensus_queue.cc:579] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.001s)
I20251023 20:17:44.465569 21454 raft_consensus.cc:1217] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251023 20:17:44.488309 21454 raft_consensus.cc:1217] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20251023 20:17:44.508492 21218 consensus_peers.cc:597] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:44.603801 21509 consensus_queue.cc:579] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.126s)
I20251023 20:17:44.610210 21327 raft_consensus.cc:1275] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Refusing update from remote peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251023 20:17:44.610682 21485 consensus_queue.cc:1048] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:17:44.624616 21454 raft_consensus.cc:1217] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20251023 20:17:44.635156 21122 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver c3b9f40f341e4b63a82759206f1e882d not available for placement
I20251023 20:17:44.868115 21508 raft_consensus.cc:1064] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: attempting to promote NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 to VOTER
I20251023 20:17:44.868762 21508 consensus_queue.cc:237] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:44.870738 21454 raft_consensus.cc:1275] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:44.871424 21252 raft_consensus.cc:1275] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:44.871395 21515 consensus_queue.cc:1048] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251023 20:17:44.871820 21293 consensus_peers.cc:597] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:44.872108 21515 consensus_queue.cc:1048] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:44.874282 21508 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.874758 21252 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.877228 21454 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.878188 21072 catalog_manager.cc:5649] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:17:44.883229 21327 consensus_queue.cc:237] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:44.884531 21252 raft_consensus.cc:1275] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:44.884545 21454 raft_consensus.cc:1275] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:44.884927 21518 consensus_queue.cc:1048] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:44.885085 21515 consensus_queue.cc:1048] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:44.886674 21518 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.887379 21252 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.889344 21454 raft_consensus.cc:2955] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.889940 21059 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4cf4d2ea7b734f6db1f467510835c352 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:44.890686 21073 catalog_manager.cc:5649] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:17:44.893185 21515 raft_consensus.cc:1064] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: attempting to promote NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 to VOTER
I20251023 20:17:44.893867 21515 consensus_queue.cc:237] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:44.896243 21453 raft_consensus.cc:1275] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:44.896757 21252 raft_consensus.cc:1275] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:44.897078 21518 consensus_queue.cc:1048] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:44.897595 21515 consensus_queue.cc:1048] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251023 20:17:44.897789 21293 consensus_peers.cc:597] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:44.900089 21518 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.901034 21252 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.904398 21072 catalog_manager.cc:5649] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff reported cstate change: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.905792 21453 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
W20251023 20:17:44.909957 21061 catalog_manager.cc:4712] TS c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4cf4d2ea7b734f6db1f467510835c352: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
I20251023 20:17:44.912768 21327 consensus_queue.cc:237] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:44.914227 21252 raft_consensus.cc:1275] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:44.914273 21453 raft_consensus.cc:1275] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:44.914604 21515 consensus_queue.cc:1048] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:44.914741 21518 consensus_queue.cc:1048] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:44.916498 21515 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.917016 21453 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.917016 21252 raft_consensus.cc:2955] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:44.919109 21059 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 0c966dadbdc14249b3b1d23723a0f469 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:44.920038 21073 catalog_manager.cc:5649] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:17:45.085458 21485 consensus_queue.cc:579] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.064s)
I20251023 20:17:45.089013 21252 consensus_queue.cc:237] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:45.090802 21327 raft_consensus.cc:1275] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Refusing update from remote peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:17:45.091393 21527 consensus_queue.cc:1048] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 20:17:45.092759 21218 consensus_peers.cc:597] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:45.093504 21511 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
I20251023 20:17:45.094103 21327 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } } }
W20251023 20:17:45.097390 21217 consensus_peers.cc:597] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff -> Peer efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196:42139): Couldn't send request to peer efe4dc319f25495580e38f8a19b2b7d4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2997cd6487c4020965aa97ceb29b288. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:45.097738 21061 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e2997cd6487c4020965aa97ceb29b288 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 20:17:45.098798 21074 catalog_manager.cc:5649] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff reported cstate change: config changed from index -1 to 2, NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) added. New cstate: current_term: 2 leader_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:45.178790 21515 raft_consensus.cc:1064] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: attempting to promote NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 to VOTER
I20251023 20:17:45.179467 21515 consensus_queue.cc:237] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:45.181430 21453 raft_consensus.cc:1275] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 LEARNER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:45.181713 21252 raft_consensus.cc:1275] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:17:45.181968 21518 consensus_queue.cc:1048] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:45.182192 21508 consensus_queue.cc:1048] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251023 20:17:45.182376 21293 consensus_peers.cc:597] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:45.184482 21518 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.185148 21453 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.188902 21252 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.189625 21074 catalog_manager.cc:5649] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 reported cstate change: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.193899 21327 consensus_queue.cc:237] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:45.195312 21453 raft_consensus.cc:1275] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:45.195313 21252 raft_consensus.cc:1275] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Refusing update from remote peer 53ede9ebe3094cf99e1a5286a164319d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:17:45.195778 21518 consensus_queue.cc:1048] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:45.195974 21508 consensus_queue.cc:1048] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:45.197762 21508 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.198407 21453 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.198443 21252 raft_consensus.cc:2955] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:45.201066 21059 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 8fba655f10914ec5a3bbdbd7dcf8ab53 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:45.201581 21074 catalog_manager.cc:5649] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 reported cstate change: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New cstate: current_term: 1 leader_uuid: "53ede9ebe3094cf99e1a5286a164319d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
W20251023 20:17:45.636040 21122 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver c3b9f40f341e4b63a82759206f1e882d not available for placement
I20251023 20:17:45.678665 21535 ts_tablet_manager.cc:933] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: Initiating tablet copy from peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff (127.15.81.194:43737)
I20251023 20:17:45.679183 21535 tablet_copy_client.cc:323] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:43737
I20251023 20:17:45.684801 21263 tablet_copy_service.cc:140] P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Received BeginTabletCopySession request for tablet e2997cd6487c4020965aa97ceb29b288 from peer efe4dc319f25495580e38f8a19b2b7d4 ({username='slave'} at 127.0.0.1:50154)
I20251023 20:17:45.684964 21263 tablet_copy_service.cc:161] P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Beginning new tablet copy session on tablet e2997cd6487c4020965aa97ceb29b288 from peer efe4dc319f25495580e38f8a19b2b7d4 at {username='slave'} at 127.0.0.1:50154: session id = efe4dc319f25495580e38f8a19b2b7d4-e2997cd6487c4020965aa97ceb29b288
I20251023 20:17:45.686932 21263 tablet_copy_source_session.cc:215] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:17:45.688022 21535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2997cd6487c4020965aa97ceb29b288. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:45.692338 21535 tablet_copy_client.cc:806] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 0 data blocks...
I20251023 20:17:45.692572 21535 tablet_copy_client.cc:670] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Starting download of 1 WAL segments...
I20251023 20:17:45.693826 21535 tablet_copy_client.cc:538] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:17:45.695868 21535 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap starting.
I20251023 20:17:45.700027 21535 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:17:45.700316 21535 tablet_bootstrap.cc:492] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: Bootstrap complete.
I20251023 20:17:45.700500 21535 ts_tablet_manager.cc:1403] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20251023 20:17:45.701139 21535 raft_consensus.cc:359] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:45.701301 21535 raft_consensus.cc:740] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: efe4dc319f25495580e38f8a19b2b7d4, State: Initialized, Role: LEARNER
I20251023 20:17:45.701480 21535 consensus_queue.cc:260] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: true } }
I20251023 20:17:45.702093 21535 ts_tablet_manager.cc:1434] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251023 20:17:45.702817 21263 tablet_copy_service.cc:342] P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Request end of tablet copy session efe4dc319f25495580e38f8a19b2b7d4-e2997cd6487c4020965aa97ceb29b288 received from {username='slave'} at 127.0.0.1:50154
I20251023 20:17:45.702922 21263 tablet_copy_service.cc:434] P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: ending tablet copy session efe4dc319f25495580e38f8a19b2b7d4-e2997cd6487c4020965aa97ceb29b288 on tablet e2997cd6487c4020965aa97ceb29b288 with peer efe4dc319f25495580e38f8a19b2b7d4
I20251023 20:17:46.063599 21453 raft_consensus.cc:1217] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
W20251023 20:17:46.069703 21061 catalog_manager.cc:4712] TS c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4cf4d2ea7b734f6db1f467510835c352: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
I20251023 20:17:46.329038 21543 consensus_queue.cc:579] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Leader has been unable to successfully communicate with peer c3b9f40f341e4b63a82759206f1e882d for more than 1 seconds (1.240s)
I20251023 20:17:46.549988 21543 raft_consensus.cc:1064] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: attempting to promote NON_VOTER efe4dc319f25495580e38f8a19b2b7d4 to VOTER
I20251023 20:17:46.550601 21543 consensus_queue.cc:237] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:46.552563 21453 raft_consensus.cc:1275] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 LEARNER]: Refusing update from remote peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:17:46.552655 21327 raft_consensus.cc:1275] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Refusing update from remote peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:17:46.553046 21543 consensus_queue.cc:1048] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:17:46.553239 21545 consensus_queue.cc:1048] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251023 20:17:46.553706 21218 consensus_peers.cc:597] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff -> Peer c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): Couldn't send request to peer c3b9f40f341e4b63a82759206f1e882d. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:17:46.555811 21545 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:46.556384 21327 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:46.560031 21453 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:46.560938 21074 catalog_manager.cc:5649] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff reported cstate change: config changed from index 2 to 3, efe4dc319f25495580e38f8a19b2b7d4 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c3b9f40f341e4b63a82759206f1e882d" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:17:46.566742 21252 consensus_queue.cc:237] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } }
I20251023 20:17:46.568051 21327 raft_consensus.cc:1275] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Refusing update from remote peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:17:46.568058 21453 raft_consensus.cc:1275] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 FOLLOWER]: Refusing update from remote peer 9ac7ad9ef0bc478d95c036f4fb6ea7ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:17:46.568446 21527 consensus_queue.cc:1048] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:46.568670 21545 consensus_queue.cc:1048] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:17:46.570179 21527 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:46.570727 21327 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:46.573037 21061 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e2997cd6487c4020965aa97ceb29b288 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:17:46.574630 21453 raft_consensus.cc:2955] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
I20251023 20:17:46.574800 21072 catalog_manager.cc:5649] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d reported cstate change: config changed from index 3 to 4, VOTER c3b9f40f341e4b63a82759206f1e882d (127.15.81.193) evicted. New cstate: current_term: 2 leader_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9ac7ad9ef0bc478d95c036f4fb6ea7ff" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 43737 } } peers { permanent_uuid: "53ede9ebe3094cf99e1a5286a164319d" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33329 } } peers { permanent_uuid: "efe4dc319f25495580e38f8a19b2b7d4" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 42139 } attrs { promote: false } } }
W20251023 20:17:47.137203 21061 catalog_manager.cc:4712] TS c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0c966dadbdc14249b3b1d23723a0f469: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
W20251023 20:17:48.813074 21061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111) [suppressed 55 similar messages]
W20251023 20:17:48.814146 21061 catalog_manager.cc:4712] TS c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e2997cd6487c4020965aa97ceb29b288: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
I20251023 20:17:50.420226 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:17:50.427316 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:50.427577 15687 tablet_replica.cc:333] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: stopping tablet replica
I20251023 20:17:50.427748 15687 raft_consensus.cc:2243] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:17:50.428035 15687 raft_consensus.cc:2272] T e2997cd6487c4020965aa97ceb29b288 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.428610 15687 tablet_replica.cc:333] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: stopping tablet replica
I20251023 20:17:50.428750 15687 raft_consensus.cc:2243] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.428977 15687 raft_consensus.cc:2272] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.429415 15687 tablet_replica.cc:333] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: stopping tablet replica
I20251023 20:17:50.429553 15687 raft_consensus.cc:2243] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.429734 15687 raft_consensus.cc:2272] T 4cf4d2ea7b734f6db1f467510835c352 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.430150 15687 tablet_replica.cc:333] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff: stopping tablet replica
I20251023 20:17:50.430269 15687 raft_consensus.cc:2243] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.430470 15687 raft_consensus.cc:2272] T 0c966dadbdc14249b3b1d23723a0f469 P 9ac7ad9ef0bc478d95c036f4fb6ea7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.454447 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:17:50.457634 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:17:50.465703 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:50.465931 15687 tablet_replica.cc:333] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d: stopping tablet replica
I20251023 20:17:50.466113 15687 raft_consensus.cc:2243] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:50.466369 15687 raft_consensus.cc:2272] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.466851 15687 tablet_replica.cc:333] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d: stopping tablet replica
I20251023 20:17:50.466953 15687 raft_consensus.cc:2243] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:50.467159 15687 raft_consensus.cc:2272] T 0c966dadbdc14249b3b1d23723a0f469 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.467521 15687 tablet_replica.cc:333] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d: stopping tablet replica
I20251023 20:17:50.467610 15687 raft_consensus.cc:2243] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.467772 15687 raft_consensus.cc:2272] T e2997cd6487c4020965aa97ceb29b288 P 53ede9ebe3094cf99e1a5286a164319d [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.468117 15687 tablet_replica.cc:333] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d: stopping tablet replica
I20251023 20:17:50.468200 15687 raft_consensus.cc:2243] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:50.468377 15687 raft_consensus.cc:2272] T 4cf4d2ea7b734f6db1f467510835c352 P 53ede9ebe3094cf99e1a5286a164319d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.481848 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:17:50.485013 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:17:50.492928 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:50.493151 15687 tablet_replica.cc:333] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4: stopping tablet replica
I20251023 20:17:50.493333 15687 raft_consensus.cc:2243] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.493502 15687 raft_consensus.cc:2272] T e2997cd6487c4020965aa97ceb29b288 P efe4dc319f25495580e38f8a19b2b7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.493907 15687 tablet_replica.cc:333] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4: stopping tablet replica
I20251023 20:17:50.494005 15687 raft_consensus.cc:2243] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.494143 15687 raft_consensus.cc:2272] T 4cf4d2ea7b734f6db1f467510835c352 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.494519 15687 tablet_replica.cc:333] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4: stopping tablet replica
I20251023 20:17:50.494606 15687 raft_consensus.cc:2243] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.494748 15687 raft_consensus.cc:2272] T 0c966dadbdc14249b3b1d23723a0f469 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.495062 15687 tablet_replica.cc:333] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4: stopping tablet replica
I20251023 20:17:50.495141 15687 raft_consensus.cc:2243] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:50.495273 15687 raft_consensus.cc:2272] T 8fba655f10914ec5a3bbdbd7dcf8ab53 P efe4dc319f25495580e38f8a19b2b7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:50.508594 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:17:50.511600 15687 master.cc:561] Master@127.15.81.254:33457 shutting down...
W20251023 20:17:50.875315 21061 catalog_manager.cc:4712] TS c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e2997cd6487c4020965aa97ceb29b288: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
W20251023 20:17:53.342698 21061 catalog_manager.cc:4712] TS c3b9f40f341e4b63a82759206f1e882d (127.15.81.193:46417): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 0c966dadbdc14249b3b1d23723a0f469: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46417: connect: Connection refused (error 111)
I20251023 20:17:54.110023 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:54.110324 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:54.110460 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P 81a0b2be187a4234ac49640233315c98: stopping tablet replica
I20251023 20:17:54.136474 15687 master.cc:583] Master@127.15.81.254:33457 shutdown complete.
[ OK ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering (13565 ms)
[ RUN ] AutoRebalancerTest.TestHandlingFailedTservers
I20251023 20:17:54.150090 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:37985
I20251023 20:17:54.150604 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:54.152786 21560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.152948 21561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.153252 21563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:54.153592 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:54.154279 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:54.154350 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:54.154407 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250674154386 us; error 0 us; skew 500 ppm
I20251023 20:17:54.154556 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:54.155584 15687 webserver.cc:492] Webserver started at http://127.15.81.254:44165/ using document root <none> and password file <none>
I20251023 20:17:54.155793 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:54.155867 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:54.156006 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:54.156620 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "bef72f756d014c56bc85a84cbce29f66"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.158941 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251023 20:17:54.160456 21568 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.160801 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:17:54.160949 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "bef72f756d014c56bc85a84cbce29f66"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.161075 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:54.199415 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:54.199823 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:54.208907 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:37985
I20251023 20:17:54.208925 21620 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:37985 every 8 connection(s)
I20251023 20:17:54.209941 21621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:54.213608 21621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66: Bootstrap starting.
I20251023 20:17:54.214548 21621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:54.215689 21621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66: No bootstrap required, opened a new log
I20251023 20:17:54.216224 21621 raft_consensus.cc:359] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bef72f756d014c56bc85a84cbce29f66" member_type: VOTER }
I20251023 20:17:54.216339 21621 raft_consensus.cc:385] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:54.216374 21621 raft_consensus.cc:740] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bef72f756d014c56bc85a84cbce29f66, State: Initialized, Role: FOLLOWER
I20251023 20:17:54.216503 21621 consensus_queue.cc:260] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bef72f756d014c56bc85a84cbce29f66" member_type: VOTER }
I20251023 20:17:54.216583 21621 raft_consensus.cc:399] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:17:54.216616 21621 raft_consensus.cc:493] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:17:54.216660 21621 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:54.218190 21621 raft_consensus.cc:515] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bef72f756d014c56bc85a84cbce29f66" member_type: VOTER }
I20251023 20:17:54.218358 21621 leader_election.cc:304] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bef72f756d014c56bc85a84cbce29f66; no voters:
I20251023 20:17:54.218652 21621 leader_election.cc:290] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:17:54.218755 21624 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:54.219202 21624 raft_consensus.cc:697] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 1 LEADER]: Becoming Leader. State: Replica: bef72f756d014c56bc85a84cbce29f66, State: Running, Role: LEADER
I20251023 20:17:54.219354 21621 sys_catalog.cc:565] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:17:54.219384 21624 consensus_queue.cc:237] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bef72f756d014c56bc85a84cbce29f66" member_type: VOTER }
I20251023 20:17:54.220096 21626 sys_catalog.cc:455] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bef72f756d014c56bc85a84cbce29f66. Latest consensus state: current_term: 1 leader_uuid: "bef72f756d014c56bc85a84cbce29f66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bef72f756d014c56bc85a84cbce29f66" member_type: VOTER } }
I20251023 20:17:54.220170 21625 sys_catalog.cc:455] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bef72f756d014c56bc85a84cbce29f66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bef72f756d014c56bc85a84cbce29f66" member_type: VOTER } }
I20251023 20:17:54.220279 21626 sys_catalog.cc:458] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:54.220376 21625 sys_catalog.cc:458] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [sys.catalog]: This master's current role is: LEADER
I20251023 20:17:54.223742 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251023 20:17:54.224438 21640 catalog_manager.cc:1568] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 20:17:54.224504 21640 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 20:17:54.224586 21631 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:17:54.225606 21631 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:17:54.227844 21631 catalog_manager.cc:1357] Generated new cluster ID: fa3beef10a51484dafce20cb8206dcf8
I20251023 20:17:54.227912 21631 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:17:54.265144 21631 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:17:54.265739 21631 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:17:54.272799 21631 catalog_manager.cc:6022] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66: Generated new TSK 0
I20251023 20:17:54.272990 21631 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:17:54.288376 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:54.290962 21642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.291119 21643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.291604 21645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:54.292052 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:54.292269 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:54.292330 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:54.292374 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250674292369 us; error 0 us; skew 500 ppm
I20251023 20:17:54.292513 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:54.293583 15687 webserver.cc:492] Webserver started at http://127.15.81.193:33445/ using document root <none> and password file <none>
I20251023 20:17:54.293800 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:54.293854 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:54.293937 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:54.294490 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "a4e6557426d044a1929898c88af7be09"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.296553 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:54.297905 21651 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.298206 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:54.298282 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "a4e6557426d044a1929898c88af7be09"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.298378 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:54.313839 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:54.314190 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:54.314631 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:54.315412 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:54.315460 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.315526 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:54.315567 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.325580 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:46685
I20251023 20:17:54.325598 21715 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:46685 every 8 connection(s)
I20251023 20:17:54.326862 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:54.329097 21720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.329317 21721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.329953 21723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:54.330724 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:54.330924 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:54.330981 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:54.331027 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250674331021 us; error 0 us; skew 500 ppm
I20251023 20:17:54.331172 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:54.332358 15687 webserver.cc:492] Webserver started at http://127.15.81.194:42093/ using document root <none> and password file <none>
I20251023 20:17:54.332556 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:54.332616 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:54.332722 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:54.333295 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "8a617d97848847c79d0559f2680f6ef8"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.335402 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 20:17:54.336834 21728 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.337399 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:54.337486 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "8a617d97848847c79d0559f2680f6ef8"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.337571 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:54.337852 21716 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:17:54.337965 21716 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:54.338340 21716 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
I20251023 20:17:54.339447 21586 ts_manager.cc:194] Registered new tserver with Master: a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685)
I20251023 20:17:54.340390 21586 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51252
I20251023 20:17:54.352569 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:54.353025 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:54.353492 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:54.354203 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:54.354249 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.354307 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:54.354348 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.365036 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:46425
I20251023 20:17:54.365053 21791 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:46425 every 8 connection(s)
I20251023 20:17:54.366355 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:54.368516 21795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.368764 21796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:54.370019 21798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:54.371762 21792 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:17:54.371866 21792 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:54.372162 21792 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
I20251023 20:17:54.372972 21586 ts_manager.cc:194] Registered new tserver with Master: 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425)
I20251023 20:17:54.373646 21586 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51264
I20251023 20:17:54.375349 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:54.375589 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:54.375643 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:54.375684 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250674375681 us; error 0 us; skew 500 ppm
I20251023 20:17:54.375808 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:54.376741 15687 webserver.cc:492] Webserver started at http://127.15.81.195:41105/ using document root <none> and password file <none>
I20251023 20:17:54.376965 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:54.377043 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:54.377166 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:54.377692 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "09b892db91c241458ecde269e5b26587"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.379884 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:54.381243 21803 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.381654 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:54.381764 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "09b892db91c241458ecde269e5b26587"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:17:54.381865 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:54.389417 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:54.389760 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:54.390177 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:54.390823 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:54.390889 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.390982 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:54.391043 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:54.401367 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:46071
I20251023 20:17:54.401387 21865 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:46071 every 8 connection(s)
I20251023 20:17:54.406888 21866 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:17:54.406999 21866 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:54.407317 21866 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
I20251023 20:17:54.407999 21586 ts_manager.cc:194] Registered new tserver with Master: 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:54.408600 21586 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51272
I20251023 20:17:54.408965 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006826847s
I20251023 20:17:55.341753 21716 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
I20251023 20:17:55.374655 21792 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
I20251023 20:17:55.409816 21866 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
I20251023 20:17:55.435812 15687 test_util.cc:276] Using random seed: 551463198
I20251023 20:17:55.447346 21586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34572:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:17:55.448042 21586 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:17:55.480254 21756 tablet_service.cc:1505] Processing CreateTablet for tablet 2c765af5165d40eabfd0dec0ef96d4fd (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:17:55.480365 21755 tablet_service.cc:1505] Processing CreateTablet for tablet 8321320ad7664bb68aaafce2f1aa341e (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:17:55.480551 21753 tablet_service.cc:1505] Processing CreateTablet for tablet 1f10a9e0110747188abf9e526a6e816c (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:17:55.480875 21756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c765af5165d40eabfd0dec0ef96d4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.481127 21753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f10a9e0110747188abf9e526a6e816c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.481158 21754 tablet_service.cc:1505] Processing CreateTablet for tablet 28185f37b35d46b7b1a4d37110fceed3 (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:17:55.481611 21754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28185f37b35d46b7b1a4d37110fceed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.485204 21831 tablet_service.cc:1505] Processing CreateTablet for tablet 2c765af5165d40eabfd0dec0ef96d4fd (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:17:55.485606 21831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c765af5165d40eabfd0dec0ef96d4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.486189 21828 tablet_service.cc:1505] Processing CreateTablet for tablet 1f10a9e0110747188abf9e526a6e816c (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:17:55.486605 21828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f10a9e0110747188abf9e526a6e816c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.486742 21679 tablet_service.cc:1505] Processing CreateTablet for tablet 2c765af5165d40eabfd0dec0ef96d4fd (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:17:55.487120 21679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c765af5165d40eabfd0dec0ef96d4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.489215 21678 tablet_service.cc:1505] Processing CreateTablet for tablet 8321320ad7664bb68aaafce2f1aa341e (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:17:55.489615 21678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8321320ad7664bb68aaafce2f1aa341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.489534 21830 tablet_service.cc:1505] Processing CreateTablet for tablet 8321320ad7664bb68aaafce2f1aa341e (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:17:55.489982 21830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8321320ad7664bb68aaafce2f1aa341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.493254 21755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8321320ad7664bb68aaafce2f1aa341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.493191 21677 tablet_service.cc:1505] Processing CreateTablet for tablet 28185f37b35d46b7b1a4d37110fceed3 (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:17:55.493551 21677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28185f37b35d46b7b1a4d37110fceed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.496744 21887 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:17:55.496702 21676 tablet_service.cc:1505] Processing CreateTablet for tablet 1f10a9e0110747188abf9e526a6e816c (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:17:55.497901 21676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f10a9e0110747188abf9e526a6e816c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.489216 21829 tablet_service.cc:1505] Processing CreateTablet for tablet 28185f37b35d46b7b1a4d37110fceed3 (DEFAULT_TABLE table=test-workload [id=7d91e5ec1f074aa89dc94d3a99c7ef74]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:17:55.497427 21886 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:17:55.501259 21829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28185f37b35d46b7b1a4d37110fceed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:17:55.502921 21886 tablet_bootstrap.cc:654] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.514928 21887 tablet_bootstrap.cc:654] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.515856 21889 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:17:55.517374 21887 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: No bootstrap required, opened a new log
I20251023 20:17:55.517428 21889 tablet_bootstrap.cc:654] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.517575 21887 ts_tablet_manager.cc:1403] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20251023 20:17:55.518481 21887 raft_consensus.cc:359] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.518695 21887 raft_consensus.cc:385] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.518788 21887 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.519060 21886 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: No bootstrap required, opened a new log
I20251023 20:17:55.519066 21887 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.519250 21889 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: No bootstrap required, opened a new log
I20251023 20:17:55.519215 21886 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.009s
I20251023 20:17:55.519410 21889 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:17:55.520108 21886 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.520187 21889 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.520367 21886 raft_consensus.cc:385] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.520464 21889 raft_consensus.cc:385] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.520413 21887 ts_tablet_manager.cc:1434] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251023 20:17:55.520535 21886 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.521464 21889 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.521682 21887 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:17:55.521777 21886 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.521869 21889 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.523612 21887 tablet_bootstrap.cc:654] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.524482 21889 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251023 20:17:55.525426 21886 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20251023 20:17:55.525918 21886 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:17:55.526523 21887 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: No bootstrap required, opened a new log
I20251023 20:17:55.526721 21887 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20251023 20:17:55.527347 21886 tablet_bootstrap.cc:654] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.527494 21887 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.527765 21887 raft_consensus.cc:385] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.527901 21887 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.528219 21887 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.528934 21889 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:17:55.529806 21887 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:17:55.530218 21887 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:17:55.530891 21889 tablet_bootstrap.cc:654] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.532999 21887 tablet_bootstrap.cc:654] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.533200 21889 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: No bootstrap required, opened a new log
I20251023 20:17:55.533344 21889 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251023 20:17:55.533962 21889 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.534229 21889 raft_consensus.cc:385] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.534339 21889 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.534631 21889 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.534889 21887 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: No bootstrap required, opened a new log
I20251023 20:17:55.535080 21887 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251023 20:17:55.535153 21886 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: No bootstrap required, opened a new log
I20251023 20:17:55.535164 21889 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:17:55.535799 21889 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:17:55.535892 21887 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.536134 21887 raft_consensus.cc:385] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.536257 21887 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.536564 21887 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.535354 21886 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.007s
I20251023 20:17:55.537384 21889 tablet_bootstrap.cc:654] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.537804 21887 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251023 20:17:55.538141 21887 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:17:55.539245 21886 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.539426 21886 raft_consensus.cc:385] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.539517 21886 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.539762 21886 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.540264 21886 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251023 20:17:55.540642 21886 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:17:55.542516 21886 tablet_bootstrap.cc:654] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.543138 21887 tablet_bootstrap.cc:654] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.546805 21887 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: No bootstrap required, opened a new log
I20251023 20:17:55.546882 21889 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: No bootstrap required, opened a new log
I20251023 20:17:55.546975 21887 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20251023 20:17:55.547032 21889 ts_tablet_manager.cc:1403] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20251023 20:17:55.547724 21889 raft_consensus.cc:359] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.547724 21887 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.547960 21889 raft_consensus.cc:385] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.547964 21887 raft_consensus.cc:385] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.548075 21889 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.548117 21887 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.548362 21887 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.548372 21889 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.548861 21889 ts_tablet_manager.cc:1434] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:17:55.549150 21889 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:17:55.550069 21887 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251023 20:17:55.550863 21889 tablet_bootstrap.cc:654] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.551456 21886 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: No bootstrap required, opened a new log
I20251023 20:17:55.551596 21886 ts_tablet_manager.cc:1403] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20251023 20:17:55.552407 21886 raft_consensus.cc:359] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.552608 21886 raft_consensus.cc:385] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.552688 21886 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.552939 21886 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.554032 21886 ts_tablet_manager.cc:1434] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:17:55.554319 21886 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:17:55.555912 21889 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: No bootstrap required, opened a new log
I20251023 20:17:55.555943 21886 tablet_bootstrap.cc:654] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Neither blocks nor log segments found. Creating new log.
I20251023 20:17:55.556087 21889 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20251023 20:17:55.556695 21889 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.556877 21889 raft_consensus.cc:385] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.556978 21889 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.557196 21889 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.557657 21889 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251023 20:17:55.561641 21886 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: No bootstrap required, opened a new log
I20251023 20:17:55.561757 21886 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20251023 20:17:55.562309 21886 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.562456 21886 raft_consensus.cc:385] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:17:55.562513 21886 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:17:55.562676 21886 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.563110 21886 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:17:55.567138 21893 raft_consensus.cc:493] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:55.567266 21893 raft_consensus.cc:515] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.567921 21893 leader_election.cc:290] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.572969 21893 raft_consensus.cc:493] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:55.573135 21893 raft_consensus.cc:515] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.573791 21893 leader_election.cc:290] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.574899 21841 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:17:55.574921 21689 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09" is_pre_election: true
I20251023 20:17:55.575039 21688 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09" is_pre_election: true
I20251023 20:17:55.575018 21840 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:17:55.575199 21841 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 0.
I20251023 20:17:55.575230 21688 raft_consensus.cc:2468] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 0.
I20251023 20:17:55.575346 21840 raft_consensus.cc:2468] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 0.
I20251023 20:17:55.575227 21689 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 0.
I20251023 20:17:55.575654 21731 leader_election.cc:304] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:17:55.575721 21732 leader_election.cc:304] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, 8a617d97848847c79d0559f2680f6ef8; no voters:
I20251023 20:17:55.576156 21893 raft_consensus.cc:2804] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:55.576309 21893 raft_consensus.cc:493] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:55.576318 21906 raft_consensus.cc:2804] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:55.576429 21893 raft_consensus.cc:3060] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.576452 21906 raft_consensus.cc:493] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:55.576556 21906 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Advancing to term 1
W20251023 20:17:55.577553 21717 tablet.cc:2378] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 20:17:55.578526 21893 raft_consensus.cc:515] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.578747 21906 raft_consensus.cc:515] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.579037 21893 leader_election.cc:290] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 election: Requested vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.579421 21689 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09"
I20251023 20:17:55.579579 21906 leader_election.cc:290] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 election: Requested vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.579643 21689 raft_consensus.cc:3060] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.579700 21688 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09"
I20251023 20:17:55.579861 21841 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:17:55.579907 21688 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.580026 21841 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.581605 21689 raft_consensus.cc:2468] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 1.
I20251023 20:17:55.581880 21840 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:17:55.582043 21840 raft_consensus.cc:3060] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.582650 21841 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 1.
I20251023 20:17:55.582904 21731 leader_election.cc:304] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:17:55.582650 21688 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 1.
I20251023 20:17:55.583083 21732 leader_election.cc:304] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, 8a617d97848847c79d0559f2680f6ef8; no voters:
I20251023 20:17:55.583436 21893 raft_consensus.cc:2804] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:55.583600 21906 raft_consensus.cc:2804] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:55.583988 21906 raft_consensus.cc:697] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 LEADER]: Becoming Leader. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:17:55.584333 21906 consensus_queue.cc:237] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.584579 21840 raft_consensus.cc:2468] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 1.
I20251023 20:17:55.584946 21893 raft_consensus.cc:697] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 LEADER]: Becoming Leader. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:17:55.585198 21893 consensus_queue.cc:237] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.587333 21584 catalog_manager.cc:5649] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "8a617d97848847c79d0559f2680f6ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:55.590754 21584 catalog_manager.cc:5649] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "8a617d97848847c79d0559f2680f6ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:55.681761 21894 raft_consensus.cc:493] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:55.681936 21894 raft_consensus.cc:515] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.682715 21894 leader_election.cc:290] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.688768 21766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8" is_pre_election: true
I20251023 20:17:55.688910 21840 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:17:55.689008 21766 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 0.
I20251023 20:17:55.689109 21840 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 0.
I20251023 20:17:55.689450 21654 leader_election.cc:304] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:17:55.689683 21894 raft_consensus.cc:2804] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:55.689774 21894 raft_consensus.cc:493] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:55.689868 21894 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.691633 21894 raft_consensus.cc:515] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.692178 21894 leader_election.cc:290] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 election: Requested vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.692512 21766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8"
I20251023 20:17:55.692656 21766 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.692801 21840 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:17:55.692986 21840 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.694742 21766 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 1.
I20251023 20:17:55.694835 21840 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 1.
I20251023 20:17:55.695089 21654 leader_election.cc:304] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:17:55.695334 21894 raft_consensus.cc:2804] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:55.695700 21894 raft_consensus.cc:697] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 LEADER]: Becoming Leader. State: Replica: a4e6557426d044a1929898c88af7be09, State: Running, Role: LEADER
I20251023 20:17:55.695933 21894 consensus_queue.cc:237] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.698192 21584 catalog_manager.cc:5649] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4e6557426d044a1929898c88af7be09 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:55.767416 21894 raft_consensus.cc:493] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:17:55.767618 21894 raft_consensus.cc:515] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.768231 21894 leader_election.cc:290] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.768601 21766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8" is_pre_election: true
I20251023 20:17:55.768679 21840 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:17:55.768782 21766 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 0.
I20251023 20:17:55.768893 21840 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 0.
I20251023 20:17:55.769161 21654 leader_election.cc:304] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:17:55.769410 21894 raft_consensus.cc:2804] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:17:55.769502 21894 raft_consensus.cc:493] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:17:55.769590 21894 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.771311 21894 raft_consensus.cc:515] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.771791 21894 leader_election.cc:290] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 election: Requested vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:17:55.772138 21766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8"
I20251023 20:17:55.772212 21840 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:17:55.772341 21766 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.772446 21840 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:17:55.774314 21766 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 1.
I20251023 20:17:55.774421 21840 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 1.
I20251023 20:17:55.774737 21654 leader_election.cc:304] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:17:55.775060 21894 raft_consensus.cc:2804] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:17:55.775199 21894 raft_consensus.cc:697] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 LEADER]: Becoming Leader. State: Replica: a4e6557426d044a1929898c88af7be09, State: Running, Role: LEADER
I20251023 20:17:55.775434 21894 consensus_queue.cc:237] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:17:55.777534 21584 catalog_manager.cc:5649] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4e6557426d044a1929898c88af7be09 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:17:55.835837 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:17:55.843958 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:55.844323 15687 tablet_replica.cc:333] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: stopping tablet replica
I20251023 20:17:55.844512 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.844674 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.845301 15687 tablet_replica.cc:333] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: stopping tablet replica
I20251023 20:17:55.845440 15687 raft_consensus.cc:2243] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:55.845635 15687 pending_rounds.cc:70] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Trying to abort 1 pending ops.
I20251023 20:17:55.845695 15687 pending_rounds.cc:77] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082767652503552 op_type: NO_OP noop_request { }
I20251023 20:17:55.845803 15687 raft_consensus.cc:2889] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:55.845902 15687 raft_consensus.cc:2272] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.846549 15687 tablet_replica.cc:333] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: stopping tablet replica
I20251023 20:17:55.846679 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.846802 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.847256 15687 tablet_replica.cc:333] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: stopping tablet replica
I20251023 20:17:55.847384 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:55.847530 15687 pending_rounds.cc:70] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Trying to abort 1 pending ops.
I20251023 20:17:55.847573 15687 pending_rounds.cc:77] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082767978037248 op_type: NO_OP noop_request { }
I20251023 20:17:55.847642 15687 raft_consensus.cc:2889] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:55.847702 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.861956 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
I20251023 20:17:55.865319 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:17:55.882591 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:55.882959 15687 tablet_replica.cc:333] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: stopping tablet replica
I20251023 20:17:55.883152 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:55.883366 15687 pending_rounds.cc:70] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Trying to abort 1 pending ops.
I20251023 20:17:55.883422 15687 pending_rounds.cc:77] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082767199391744 op_type: NO_OP noop_request { }
I20251023 20:17:55.883527 15687 raft_consensus.cc:2889] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:55.883610 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.884292 15687 tablet_replica.cc:333] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: stopping tablet replica
I20251023 20:17:55.884418 15687 raft_consensus.cc:2243] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.884541 15687 raft_consensus.cc:2272] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.885102 15687 tablet_replica.cc:333] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: stopping tablet replica
I20251023 20:17:55.885222 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:17:55.885386 15687 pending_rounds.cc:70] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Trying to abort 1 pending ops.
I20251023 20:17:55.885428 15687 pending_rounds.cc:77] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082767196897280 op_type: NO_OP noop_request { }
I20251023 20:17:55.885517 15687 raft_consensus.cc:2889] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:17:55.885608 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.886294 15687 tablet_replica.cc:333] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: stopping tablet replica
I20251023 20:17:55.886411 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.886521 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.891031 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:17:55.894073 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
I20251023 20:17:55.904171 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:17:55.904539 15687 tablet_replica.cc:333] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: stopping tablet replica
I20251023 20:17:55.904672 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.904790 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.905284 15687 tablet_replica.cc:333] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: stopping tablet replica
I20251023 20:17:55.905418 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.905534 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.905961 15687 tablet_replica.cc:333] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: stopping tablet replica
I20251023 20:17:55.906081 15687 raft_consensus.cc:2243] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.906198 15687 raft_consensus.cc:2272] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.906597 15687 tablet_replica.cc:333] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: stopping tablet replica
I20251023 20:17:55.906708 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:17:55.906808 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:17:55.920128 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:17:55.924000 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:17:55.926301 21916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:55.926590 21918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:17:55.926930 21920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:17:55.927459 15687 server_base.cc:1047] running on GCE node
I20251023 20:17:55.927675 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:17:55.927734 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:17:55.927778 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250675927775 us; error 0 us; skew 500 ppm
I20251023 20:17:55.927893 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:17:55.928953 15687 webserver.cc:492] Webserver started at http://127.15.81.196:38147/ using document root <none> and password file <none>
I20251023 20:17:55.929170 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:17:55.929236 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:17:55.929360 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:17:55.929900 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "16938db5e56d4e518736bca3efe91dae"
format_stamp: "Formatted at 2025-10-23 20:17:55 on dist-test-slave-jr94"
I20251023 20:17:55.932284 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:17:55.933727 21925 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:55.934080 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:17:55.934188 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "16938db5e56d4e518736bca3efe91dae"
format_stamp: "Formatted at 2025-10-23 20:17:55 on dist-test-slave-jr94"
I20251023 20:17:55.934294 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:17:55.945083 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:17:55.945510 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:17:55.945925 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:17:55.946671 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:17:55.946729 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:55.946815 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:17:55.946869 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:17:55.957270 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:33685
I20251023 20:17:55.957314 21987 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:33685 every 8 connection(s)
I20251023 20:17:55.963505 21988 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:17:55.963644 21988 heartbeater.cc:461] Registering TS with master...
I20251023 20:17:55.963925 21988 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
I20251023 20:17:55.964737 21584 ts_manager.cc:194] Registered new tserver with Master: 16938db5e56d4e518736bca3efe91dae (127.15.81.196:33685)
I20251023 20:17:55.965530 21584 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34578
W20251023 20:17:56.225840 21634 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:56.226925 21572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20251023 20:17:56.227860 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:56.227945 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:56.229251 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:56.229313 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:56.230501 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:56.230562 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
I20251023 20:17:56.966693 21988 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
W20251023 20:17:57.234323 21634 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:57.236212 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:57.236284 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:57.237572 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:57.237634 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:57.238891 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:57.238947 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:58.242995 21634 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:58.245918 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:58.246009 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:58.247355 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:58.247418 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:58.248610 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:58.248664 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:59.252211 21634 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:59.257858 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:59.257951 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:59.259294 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:59.259353 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:17:59.260535 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:17:59.260609 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:18:00.264300 21634 auto_rebalancer.cc:254] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:18:00.269762 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:18:00.269837 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.194:46425: connect: Connection refused (error 111)
W20251023 20:18:00.271155 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:18:00.271216 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:18:00.272455 21634 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:18:00.272512 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.81.193:46685: connect: Connection refused (error 111)
W20251023 20:18:01.273294 21634 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver a4e6557426d044a1929898c88af7be09 not available for placement
I20251023 20:18:01.480129 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:01.482919 22007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:01.483141 22008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:01.493635 15687 server_base.cc:1047] running on GCE node
W20251023 20:18:01.493872 22010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:01.494467 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:01.494539 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:01.494580 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250681494566 us; error 0 us; skew 500 ppm
I20251023 20:18:01.494712 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:01.495754 15687 webserver.cc:492] Webserver started at http://127.15.81.193:33445/ using document root <none> and password file <none>
I20251023 20:18:01.495950 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:01.496002 15687 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 20:18:01.497828 15687 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:01.499179 22015 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:01.499519 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:01.499634 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "a4e6557426d044a1929898c88af7be09"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:18:01.499758 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:01.519835 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:01.520423 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:01.520988 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:01.522203 22022 ts_tablet_manager.cc:548] Loading tablet metadata (0/4 complete)
I20251023 20:18:01.530037 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251023 20:18:01.530114 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.000s
I20251023 20:18:01.530228 15687 ts_tablet_manager.cc:600] Registering tablets (0/4 complete)
I20251023 20:18:01.532306 22022 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:18:01.535882 22022 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.536199 22022 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Bootstrap complete.
I20251023 20:18:01.536449 22022 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:18:01.537068 22022 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.537298 22022 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.537546 22022 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.537822 15687 ts_tablet_manager.cc:616] Registered 4 tablets
I20251023 20:18:01.537936 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.008s user 0.007s sys 0.000s
I20251023 20:18:01.538328 22022 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 20:18:01.538604 22022 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:18:01.541527 22022 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.541899 22022 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Bootstrap complete.
I20251023 20:18:01.542157 22022 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:18:01.542759 22022 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.542940 22022 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.543205 22022 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.543699 22022 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:01.543948 22022 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:18:01.547544 22022 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 20:18:01.548177 22022 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Bootstrap complete.
I20251023 20:18:01.548405 22022 ts_tablet_manager.cc:1403] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251023 20:18:01.549115 22022 raft_consensus.cc:359] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.549368 22022 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.549594 22022 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.549927 22022 ts_tablet_manager.cc:1434] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:18:01.550199 22022 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Bootstrap starting.
I20251023 20:18:01.553891 22022 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 20:18:01.554422 22022 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Bootstrap complete.
I20251023 20:18:01.554636 22022 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251023 20:18:01.555387 22022 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.557981 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:46685
I20251023 20:18:01.558024 22084 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:46685 every 8 connection(s)
I20251023 20:18:01.559468 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:01.573860 22089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:01.575023 22090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:01.578790 22085 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:18:01.578943 22085 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:01.579416 22085 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
W20251023 20:18:01.581799 22092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:01.584566 22022 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e6557426d044a1929898c88af7be09, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.584920 22022 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.585506 22022 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Time spent starting tablet: real 0.031s user 0.025s sys 0.007s
I20251023 20:18:01.585790 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:01.586019 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:01.586107 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:01.586145 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250681586142 us; error 0 us; skew 500 ppm
I20251023 20:18:01.586290 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:01.586753 21584 ts_manager.cc:194] Re-registered known tserver with Master: a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685)
I20251023 20:18:01.587464 15687 webserver.cc:492] Webserver started at http://127.15.81.194:42093/ using document root <none> and password file <none>
I20251023 20:18:01.587653 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:01.587698 15687 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 20:18:01.588438 21584 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34590
I20251023 20:18:01.589340 15687 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:01.589609 22085 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
I20251023 20:18:01.590986 22097 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:01.591339 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:01.591421 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "8a617d97848847c79d0559f2680f6ef8"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:18:01.591512 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:01.602921 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:01.603334 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:01.603754 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:01.611486 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251023 20:18:01.611557 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20251023 20:18:01.613255 22104 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:18:01.616431 22104 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 20:18:01.616744 22104 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Bootstrap complete.
I20251023 20:18:01.616947 22104 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 20:18:01.617338 15687 ts_tablet_manager.cc:616] Registered 4 tablets
I20251023 20:18:01.617395 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.005s sys 0.000s
I20251023 20:18:01.617419 22104 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.617652 22104 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.617873 22104 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.618505 22104 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:01.618703 22104 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:18:01.622725 22104 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 20:18:01.623121 22104 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Bootstrap complete.
I20251023 20:18:01.623358 22104 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:18:01.623991 22104 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.624200 22104 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.624406 22104 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.624768 22104 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:18:01.624965 22104 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:18:01.628068 22104 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.628433 22104 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Bootstrap complete.
I20251023 20:18:01.628696 22104 ts_tablet_manager.cc:1403] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:18:01.629432 22104 raft_consensus.cc:359] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.629616 22104 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.629850 22104 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.630393 22104 ts_tablet_manager.cc:1434] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:18:01.630648 22104 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Bootstrap starting.
I20251023 20:18:01.635344 22104 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.635708 22104 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Bootstrap complete.
I20251023 20:18:01.635958 22104 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20251023 20:18:01.636606 22104 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.636785 22104 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.637018 22104 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.637516 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:46425
I20251023 20:18:01.637533 22165 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:46425 every 8 connection(s)
I20251023 20:18:01.637539 22104 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:01.638622 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:01.641052 22169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:01.641129 22170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:01.646121 22172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:01.646842 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:01.647070 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:01.647122 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:01.647156 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250681647152 us; error 0 us; skew 500 ppm
I20251023 20:18:01.647287 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:01.648586 15687 webserver.cc:492] Webserver started at http://127.15.81.195:41105/ using document root <none> and password file <none>
I20251023 20:18:01.648800 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:01.648880 15687 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 20:18:01.650012 22166 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:18:01.650141 22166 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:01.650584 22166 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
I20251023 20:18:01.650856 15687 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 20:18:01.651890 21584 ts_manager.cc:194] Re-registered known tserver with Master: 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425)
I20251023 20:18:01.652618 22177 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:01.653000 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:01.653118 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "09b892db91c241458ecde269e5b26587"
format_stamp: "Formatted at 2025-10-23 20:17:54 on dist-test-slave-jr94"
I20251023 20:18:01.653211 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:01.653915 21584 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34604
I20251023 20:18:01.681677 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:01.682132 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:01.682623 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:01.690465 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251023 20:18:01.690536 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20251023 20:18:01.692265 22185 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:18:01.695818 22185 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.696125 22185 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Bootstrap complete.
I20251023 20:18:01.696316 22185 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:18:01.696676 15687 ts_tablet_manager.cc:616] Registered 4 tablets
I20251023 20:18:01.696732 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.003s sys 0.000s
I20251023 20:18:01.696867 22185 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.697100 22185 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.697340 22185 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.698014 22185 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:18:01.698230 22185 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:18:01.702329 22185 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.702665 22185 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Bootstrap complete.
I20251023 20:18:01.702956 22185 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251023 20:18:01.703603 22185 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.703733 22185 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.703943 22185 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.704319 22185 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:18:01.704473 22185 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:18:01.708181 22185 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.708470 22185 tablet_bootstrap.cc:492] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Bootstrap complete.
I20251023 20:18:01.708688 22185 ts_tablet_manager.cc:1403] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:18:01.709314 22185 raft_consensus.cc:359] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.709430 22185 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.709637 22185 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.710036 22185 ts_tablet_manager.cc:1434] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251023 20:18:01.710237 22185 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Bootstrap starting.
I20251023 20:18:01.714438 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:46071
I20251023 20:18:01.714461 22246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:46071 every 8 connection(s)
I20251023 20:18:01.714879 22185 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:01.715224 22185 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Bootstrap complete.
I20251023 20:18:01.715468 22185 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20251023 20:18:01.715992 22185 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.716149 22185 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Initialized, Role: FOLLOWER
I20251023 20:18:01.716413 22185 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:01.716892 22185 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20251023 20:18:01.720366 22247 heartbeater.cc:344] Connected to a master server at 127.15.81.254:37985
I20251023 20:18:01.720495 22247 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:01.720786 22247 heartbeater.cc:507] Master 127.15.81.254:37985 requested a full tablet report, sending...
I20251023 20:18:01.721827 21584 ts_manager.cc:194] Re-registered known tserver with Master: 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:01.723528 21584 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34608
I20251023 20:18:01.724455 22247 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
W20251023 20:18:02.281378 21634 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Replica 8a617d97848847c79d0559f2680f6ef8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } }
W20251023 20:18:02.282680 21634 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 1f10a9e0110747188abf9e526a6e816c, TS a4e6557426d044a1929898c88af7be09 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:02.282774 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 1f10a9e0110747188abf9e526a6e816c, TS a4e6557426d044a1929898c88af7be09 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:02.288080 21634 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 2c765af5165d40eabfd0dec0ef96d4fd, TS 09b892db91c241458ecde269e5b26587 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:02.288156 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 2c765af5165d40eabfd0dec0ef96d4fd, TS 09b892db91c241458ecde269e5b26587 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:02.289242 21634 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 28185f37b35d46b7b1a4d37110fceed3, TS 8a617d97848847c79d0559f2680f6ef8 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:02.289361 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 28185f37b35d46b7b1a4d37110fceed3, TS 8a617d97848847c79d0559f2680f6ef8 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.655424 22166 heartbeater.cc:499] Master 127.15.81.254:37985 was elected leader, sending a full tablet report...
I20251023 20:18:02.696597 22253 raft_consensus.cc:493] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:02.696791 22253 raft_consensus.cc:515] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.697544 22253 leader_election.cc:290] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:02.703509 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8" is_pre_election: true
I20251023 20:18:02.703583 22221 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:18:02.703926 22140 raft_consensus.cc:2410] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4e6557426d044a1929898c88af7be09 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.703891 22221 raft_consensus.cc:2468] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 1.
I20251023 20:18:02.704412 22019 leader_election.cc:304] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:18:02.704720 22253 raft_consensus.cc:2804] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:02.704856 22253 raft_consensus.cc:493] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:02.704943 22253 raft_consensus.cc:3060] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.706745 22253 raft_consensus.cc:515] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.707214 22253 leader_election.cc:290] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 2 election: Requested vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:02.707614 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8"
I20251023 20:18:02.707614 22221 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8321320ad7664bb68aaafce2f1aa341e" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:18:02.707823 22140 raft_consensus.cc:3060] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.707844 22221 raft_consensus.cc:3060] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.709717 22140 raft_consensus.cc:2410] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a4e6557426d044a1929898c88af7be09 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.709728 22221 raft_consensus.cc:2468] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 2.
I20251023 20:18:02.710125 22019 leader_election.cc:304] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:18:02.710363 22253 raft_consensus.cc:2804] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:18:02.710700 22253 raft_consensus.cc:697] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 2 LEADER]: Becoming Leader. State: Replica: a4e6557426d044a1929898c88af7be09, State: Running, Role: LEADER
I20251023 20:18:02.710922 22253 consensus_queue.cc:237] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.713187 21584 catalog_manager.cc:5649] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 reported cstate change: term changed from 1 to 2, leader changed from 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194) to a4e6557426d044a1929898c88af7be09 (127.15.81.193). New cstate: current_term: 2 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:02.872155 22263 raft_consensus.cc:493] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:02.872349 22263 raft_consensus.cc:515] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.873087 22263 leader_election.cc:290] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:02.876120 22263 raft_consensus.cc:493] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:02.876253 22263 raft_consensus.cc:515] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.876827 22263 leader_election.cc:290] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:02.879073 22059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09" is_pre_election: true
I20251023 20:18:02.879204 22221 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:18:02.879186 22058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09" is_pre_election: true
I20251023 20:18:02.879338 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:18:02.879446 22059 raft_consensus.cc:2410] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a617d97848847c79d0559f2680f6ef8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.879535 22219 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 1.
I20251023 20:18:02.879583 22058 raft_consensus.cc:2410] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a617d97848847c79d0559f2680f6ef8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.879458 22221 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 1.
I20251023 20:18:02.879931 22101 leader_election.cc:304] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, 8a617d97848847c79d0559f2680f6ef8; no voters:
I20251023 20:18:02.880193 22263 raft_consensus.cc:2804] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:02.880290 22263 raft_consensus.cc:493] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:02.880285 22101 leader_election.cc:304] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b892db91c241458ecde269e5b26587, 8a617d97848847c79d0559f2680f6ef8; no voters: a4e6557426d044a1929898c88af7be09
I20251023 20:18:02.880441 22263 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.880928 22267 raft_consensus.cc:2804] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:02.881069 22267 raft_consensus.cc:493] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:02.881201 22267 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.882671 22263 raft_consensus.cc:515] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.883184 22263 leader_election.cc:290] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 election: Requested vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:02.883291 22267 raft_consensus.cc:515] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.883598 22058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09"
I20251023 20:18:02.883658 22221 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:18:02.883827 22058 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.883836 22221 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.883932 22267 leader_election.cc:290] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 election: Requested vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:02.884351 22059 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09"
I20251023 20:18:02.884488 22059 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.886945 22059 raft_consensus.cc:2410] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a617d97848847c79d0559f2680f6ef8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.886970 22221 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 2.
I20251023 20:18:02.887270 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:18:02.887388 22101 leader_election.cc:304] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, 8a617d97848847c79d0559f2680f6ef8; no voters:
I20251023 20:18:02.887442 22219 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.887581 22058 raft_consensus.cc:2410] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a617d97848847c79d0559f2680f6ef8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.887656 22267 raft_consensus.cc:2804] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:18:02.887797 22267 raft_consensus.cc:697] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:18:02.888350 22267 consensus_queue.cc:237] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.890229 22219 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 2.
I20251023 20:18:02.890604 22101 leader_election.cc:304] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b892db91c241458ecde269e5b26587, 8a617d97848847c79d0559f2680f6ef8; no voters: a4e6557426d044a1929898c88af7be09
I20251023 20:18:02.890964 22267 raft_consensus.cc:2804] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:18:02.891126 22267 raft_consensus.cc:697] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:18:02.891157 21584 catalog_manager.cc:5649] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: term changed from 1 to 2, leader changed from a4e6557426d044a1929898c88af7be09 (127.15.81.193) to 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "8a617d97848847c79d0559f2680f6ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:02.891420 22267 consensus_queue.cc:237] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.895175 21584 catalog_manager.cc:5649] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: term changed from 1 to 2, leader changed from a4e6557426d044a1929898c88af7be09 (127.15.81.193) to 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "8a617d97848847c79d0559f2680f6ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:02.921161 22270 raft_consensus.cc:493] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:02.921350 22270 raft_consensus.cc:515] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.922107 22270 leader_election.cc:290] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425)
I20251023 20:18:02.928082 22058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "09b892db91c241458ecde269e5b26587" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09" is_pre_election: true
I20251023 20:18:02.928210 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "09b892db91c241458ecde269e5b26587" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8" is_pre_election: true
I20251023 20:18:02.928296 22058 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b892db91c241458ecde269e5b26587 in term 1.
I20251023 20:18:02.928462 22140 raft_consensus.cc:2410] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09b892db91c241458ecde269e5b26587 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.928692 22180 leader_election.cc:304] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:18:02.928958 22270 raft_consensus.cc:2804] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:02.929076 22270 raft_consensus.cc:493] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:02.929181 22270 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.931138 22270 raft_consensus.cc:515] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.931667 22270 leader_election.cc:290] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [CANDIDATE]: Term 2 election: Requested vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425)
I20251023 20:18:02.932034 22058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "09b892db91c241458ecde269e5b26587" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09"
I20251023 20:18:02.932080 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "09b892db91c241458ecde269e5b26587" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8"
I20251023 20:18:02.932174 22058 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.932253 22140 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:02.934521 22140 raft_consensus.cc:2410] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 09b892db91c241458ecde269e5b26587 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251023 20:18:02.934515 22058 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b892db91c241458ecde269e5b26587 in term 2.
I20251023 20:18:02.934953 22180 leader_election.cc:304] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters:
I20251023 20:18:02.935220 22270 raft_consensus.cc:2804] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:18:02.935567 22270 raft_consensus.cc:697] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 2 LEADER]: Becoming Leader. State: Replica: 09b892db91c241458ecde269e5b26587, State: Running, Role: LEADER
I20251023 20:18:02.935822 22270 consensus_queue.cc:237] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:02.938163 21584 catalog_manager.cc:5649] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 reported cstate change: term changed from 1 to 2, leader changed from 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194) to 09b892db91c241458ecde269e5b26587 (127.15.81.195). New cstate: current_term: 2 leader_uuid: "09b892db91c241458ecde269e5b26587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: HEALTHY } } }
I20251023 20:18:03.018399 22267 raft_consensus.cc:493] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:03.018599 22267 raft_consensus.cc:515] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.019142 22267 leader_election.cc:290] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:03.019637 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:18:03.019637 22058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09" is_pre_election: true
I20251023 20:18:03.019902 22058 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 2.
I20251023 20:18:03.020246 22100 leader_election.cc:304] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters: 09b892db91c241458ecde269e5b26587
I20251023 20:18:03.020516 22267 raft_consensus.cc:2804] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 20:18:03.020610 22267 raft_consensus.cc:493] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:03.020697 22267 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.022702 22267 raft_consensus.cc:515] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.023228 22267 leader_election.cc:290] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 3 election: Requested vote from peers a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:03.023648 22058 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4e6557426d044a1929898c88af7be09"
I20251023 20:18:03.023653 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1f10a9e0110747188abf9e526a6e816c" candidate_uuid: "8a617d97848847c79d0559f2680f6ef8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:18:03.023818 22058 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.025830 22058 raft_consensus.cc:2468] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a617d97848847c79d0559f2680f6ef8 in term 3.
I20251023 20:18:03.026171 22100 leader_election.cc:304] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a617d97848847c79d0559f2680f6ef8, a4e6557426d044a1929898c88af7be09; no voters: 09b892db91c241458ecde269e5b26587
I20251023 20:18:03.026404 22267 raft_consensus.cc:2804] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:18:03.026540 22267 raft_consensus.cc:697] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 LEADER]: Becoming Leader. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:18:03.026767 22267 consensus_queue.cc:237] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.029060 21584 catalog_manager.cc:5649] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: term changed from 2 to 3, leader changed from 09b892db91c241458ecde269e5b26587 (127.15.81.195) to 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194). New cstate: current_term: 3 leader_uuid: "8a617d97848847c79d0559f2680f6ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:03.088750 22140 raft_consensus.cc:1275] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20251023 20:18:03.088912 22140 pending_rounds.cc:85] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Aborting all ops after (but not including) 0
I20251023 20:18:03.088976 22140 pending_rounds.cc:107] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20251023 20:18:03.089025 22140 raft_consensus.cc:2889] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 20:18:03.089536 22253 consensus_queue.cc:1048] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.092950 22219 raft_consensus.cc:1275] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251023 20:18:03.093570 22257 consensus_queue.cc:1048] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.246441 22257 raft_consensus.cc:493] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:03.246640 22257 raft_consensus.cc:515] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.247270 22257 leader_election.cc:290] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:03.247821 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8" is_pre_election: true
I20251023 20:18:03.247829 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:18:03.248126 22219 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 2.
I20251023 20:18:03.248347 22257 raft_consensus.cc:493] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:03.248490 22019 leader_election.cc:304] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters: 8a617d97848847c79d0559f2680f6ef8
I20251023 20:18:03.248512 22257 raft_consensus.cc:515] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.248776 22253 raft_consensus.cc:2804] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 20:18:03.248900 22253 raft_consensus.cc:493] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:03.249010 22253 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.249150 22257 leader_election.cc:290] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:03.249450 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8" is_pre_election: true
I20251023 20:18:03.249639 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587" is_pre_election: true
I20251023 20:18:03.249850 22219 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 2.
I20251023 20:18:03.250157 22019 leader_election.cc:304] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters: 8a617d97848847c79d0559f2680f6ef8
I20251023 20:18:03.250389 22257 raft_consensus.cc:2804] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 20:18:03.250497 22257 raft_consensus.cc:493] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:03.250594 22257 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.251398 22253 raft_consensus.cc:515] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.251891 22253 leader_election.cc:290] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 election: Requested vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:03.252206 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8"
I20251023 20:18:03.252267 22219 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28185f37b35d46b7b1a4d37110fceed3" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:18:03.252386 22219 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.252614 22257 raft_consensus.cc:515] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.253221 22257 leader_election.cc:290] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 election: Requested vote from peers 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425), 09b892db91c241458ecde269e5b26587 (127.15.81.195:46071)
I20251023 20:18:03.253535 22140 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a617d97848847c79d0559f2680f6ef8"
I20251023 20:18:03.253662 22221 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c765af5165d40eabfd0dec0ef96d4fd" candidate_uuid: "a4e6557426d044a1929898c88af7be09" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "09b892db91c241458ecde269e5b26587"
I20251023 20:18:03.253849 22221 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.254977 22219 raft_consensus.cc:2468] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 3.
I20251023 20:18:03.255263 22019 leader_election.cc:304] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters: 8a617d97848847c79d0559f2680f6ef8
I20251023 20:18:03.255750 22221 raft_consensus.cc:2468] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e6557426d044a1929898c88af7be09 in term 3.
I20251023 20:18:03.255990 22257 raft_consensus.cc:2804] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:18:03.256094 22019 leader_election.cc:304] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b892db91c241458ecde269e5b26587, a4e6557426d044a1929898c88af7be09; no voters: 8a617d97848847c79d0559f2680f6ef8
I20251023 20:18:03.256151 22257 raft_consensus.cc:697] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 3 LEADER]: Becoming Leader. State: Replica: a4e6557426d044a1929898c88af7be09, State: Running, Role: LEADER
I20251023 20:18:03.256345 22253 raft_consensus.cc:2804] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 20:18:03.256429 22257 consensus_queue.cc:237] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.256740 22253 raft_consensus.cc:697] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 3 LEADER]: Becoming Leader. State: Replica: a4e6557426d044a1929898c88af7be09, State: Running, Role: LEADER
I20251023 20:18:03.256989 22253 consensus_queue.cc:237] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.258852 21584 catalog_manager.cc:5649] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 reported cstate change: term changed from 2 to 3, leader changed from 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194) to a4e6557426d044a1929898c88af7be09 (127.15.81.193). New cstate: current_term: 3 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:03.262550 21584 catalog_manager.cc:5649] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 reported cstate change: term changed from 2 to 3, leader changed from 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194) to a4e6557426d044a1929898c88af7be09 (127.15.81.193). New cstate: current_term: 3 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:03.283931 22058 raft_consensus.cc:1240] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 3 LEADER]: Rejecting Update request from peer 8a617d97848847c79d0559f2680f6ef8 for earlier term 2. Current term is 3. Ops: []
I20251023 20:18:03.284502 22267 consensus_queue.cc:1059] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.284724 22267 consensus_queue.cc:1243] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Peer a4e6557426d044a1929898c88af7be09 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 20:18:03.284821 22263 raft_consensus.cc:3055] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:18:03.284945 22263 raft_consensus.cc:740] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:18:03.285212 22263 consensus_queue.cc:260] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.285490 22263 raft_consensus.cc:3060] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Advancing to term 3
W20251023 20:18:03.292685 21634 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251023 20:18:03.293959 21634 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 1f10a9e0110747188abf9e526a6e816c, TS a4e6557426d044a1929898c88af7be09 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:03.294044 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 1f10a9e0110747188abf9e526a6e816c, TS a4e6557426d044a1929898c88af7be09 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:03.295142 21634 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 2c765af5165d40eabfd0dec0ef96d4fd, TS 8a617d97848847c79d0559f2680f6ef8 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:03.295209 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 2c765af5165d40eabfd0dec0ef96d4fd, TS 8a617d97848847c79d0559f2680f6ef8 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.296023 22221 raft_consensus.cc:1240] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Rejecting Update request from peer 8a617d97848847c79d0559f2680f6ef8 for earlier term 2. Current term is 3. Ops: []
W20251023 20:18:03.296476 21634 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 28185f37b35d46b7b1a4d37110fceed3, TS 09b892db91c241458ecde269e5b26587 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
W20251023 20:18:03.296558 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 28185f37b35d46b7b1a4d37110fceed3, TS 09b892db91c241458ecde269e5b26587 -> TS 16938db5e56d4e518736bca3efe91dae move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.296474 22263 consensus_queue.cc:1059] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.296761 22263 raft_consensus.cc:3055] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:18:03.296835 22263 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a617d97848847c79d0559f2680f6ef8, State: Running, Role: LEADER
I20251023 20:18:03.297115 22263 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.297351 22263 raft_consensus.cc:3060] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.344223 22140 raft_consensus.cc:1240] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 LEADER]: Rejecting Update request from peer 09b892db91c241458ecde269e5b26587 for earlier term 2. Current term is 3. Ops: []
I20251023 20:18:03.344854 22270 consensus_queue.cc:1059] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.345072 22270 consensus_queue.cc:1243] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [LEADER]: Peer 8a617d97848847c79d0559f2680f6ef8 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 20:18:03.345175 22273 raft_consensus.cc:3055] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 20:18:03.345250 22273 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09b892db91c241458ecde269e5b26587, State: Running, Role: LEADER
I20251023 20:18:03.345506 22273 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } }
I20251023 20:18:03.345741 22273 raft_consensus.cc:3060] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Advancing to term 3
I20251023 20:18:03.459589 22058 raft_consensus.cc:1275] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Refusing update from remote peer 8a617d97848847c79d0559f2680f6ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 20:18:03.460124 22263 consensus_queue.cc:1048] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.464582 22221 raft_consensus.cc:1275] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Refusing update from remote peer 8a617d97848847c79d0559f2680f6ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 20:18:03.465221 22267 consensus_queue.cc:1048] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.465444 22267 consensus_queue.cc:1243] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Peer 09b892db91c241458ecde269e5b26587 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 20:18:03.466336 22221 pending_rounds.cc:85] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Aborting all ops after (but not including) 0
I20251023 20:18:03.466434 22221 pending_rounds.cc:107] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251023 20:18:03.466518 22221 raft_consensus.cc:2889] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 20:18:03.668493 22140 raft_consensus.cc:1275] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 20:18:03.669075 22257 consensus_queue.cc:1048] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.669340 22257 consensus_queue.cc:1243] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [LEADER]: Peer 8a617d97848847c79d0559f2680f6ef8 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 20:18:03.670154 22140 pending_rounds.cc:85] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Aborting all ops after (but not including) 0
I20251023 20:18:03.670217 22140 pending_rounds.cc:107] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251023 20:18:03.670306 22140 raft_consensus.cc:2889] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 20:18:03.672338 22221 raft_consensus.cc:1275] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 20:18:03.672768 22281 consensus_queue.cc:1048] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.723786 22221 raft_consensus.cc:1275] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 20:18:03.724330 22281 consensus_queue.cc:1048] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.728070 22140 raft_consensus.cc:1275] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251023 20:18:03.728562 22281 consensus_queue.cc:1048] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:03.728775 22281 consensus_queue.cc:1243] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Peer 8a617d97848847c79d0559f2680f6ef8 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251023 20:18:03.729781 22140 pending_rounds.cc:85] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Aborting all ops after (but not including) 0
I20251023 20:18:03.729874 22140 pending_rounds.cc:107] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251023 20:18:03.729970 22140 raft_consensus.cc:2889] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251023 20:18:04.300217 22140 consensus_queue.cc:237] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.302313 22221 raft_consensus.cc:1275] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Refusing update from remote peer 8a617d97848847c79d0559f2680f6ef8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 20:18:04.302670 22058 raft_consensus.cc:1275] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Refusing update from remote peer 8a617d97848847c79d0559f2680f6ef8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 20:18:04.303045 22290 consensus_queue.cc:1048] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:04.303463 22267 consensus_queue.cc:1048] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:04.305625 22290 raft_consensus.cc:2955] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.306196 22221 raft_consensus.cc:2955] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.307449 22058 raft_consensus.cc:2955] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.310997 21585 catalog_manager.cc:5649] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New cstate: current_term: 3 leader_uuid: "8a617d97848847c79d0559f2680f6ef8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 20:18:04.311463 22100 consensus_peers.cc:597] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 -> Peer 16938db5e56d4e518736bca3efe91dae (127.15.81.196:33685): Couldn't send request to peer 16938db5e56d4e518736bca3efe91dae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f10a9e0110747188abf9e526a6e816c. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:04.313236 22058 consensus_queue.cc:237] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.316120 22140 raft_consensus.cc:1275] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:18:04.316428 22221 raft_consensus.cc:1275] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:18:04.317176 22292 consensus_queue.cc:1048] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:04.317745 22291 consensus_queue.cc:1048] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:04.322494 22291 raft_consensus.cc:2955] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.327039 22140 raft_consensus.cc:2955] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.330925 21584 catalog_manager.cc:5649] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 reported cstate change: config changed from index -1 to 2, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New cstate: current_term: 2 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:04.333379 22221 raft_consensus.cc:2955] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
W20251023 20:18:04.334825 22018 consensus_peers.cc:597] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 -> Peer 16938db5e56d4e518736bca3efe91dae (127.15.81.196:33685): Couldn't send request to peer 16938db5e56d4e518736bca3efe91dae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8321320ad7664bb68aaafce2f1aa341e. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:04.335566 22058 consensus_queue.cc:237] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.337752 22219 raft_consensus.cc:1275] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20251023 20:18:04.338407 22018 consensus_peers.cc:597] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 -> Peer 16938db5e56d4e518736bca3efe91dae (127.15.81.196:33685): Couldn't send request to peer 16938db5e56d4e518736bca3efe91dae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28185f37b35d46b7b1a4d37110fceed3. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:04.338573 22140 raft_consensus.cc:1275] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Refusing update from remote peer a4e6557426d044a1929898c88af7be09: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 20:18:04.339037 22291 consensus_queue.cc:1048] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:04.339682 22291 consensus_queue.cc:1048] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:04.340833 22305 raft_consensus.cc:2955] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.342011 22140 raft_consensus.cc:2955] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.343986 22221 raft_consensus.cc:2955] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.346251 21586 catalog_manager.cc:5649] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 reported cstate change: config changed from index -1 to 3, NON_VOTER 16938db5e56d4e518736bca3efe91dae (127.15.81.196) added. New cstate: current_term: 3 leader_uuid: "a4e6557426d044a1929898c88af7be09" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } } }
I20251023 20:18:04.878908 22306 ts_tablet_manager.cc:933] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: Initiating tablet copy from peer a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685)
I20251023 20:18:04.879523 22306 tablet_copy_client.cc:323] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:46685
I20251023 20:18:04.886358 22070 tablet_copy_service.cc:140] P a4e6557426d044a1929898c88af7be09: Received BeginTabletCopySession request for tablet 28185f37b35d46b7b1a4d37110fceed3 from peer 16938db5e56d4e518736bca3efe91dae ({username='slave'} at 127.0.0.1:50444)
I20251023 20:18:04.886497 22070 tablet_copy_service.cc:161] P a4e6557426d044a1929898c88af7be09: Beginning new tablet copy session on tablet 28185f37b35d46b7b1a4d37110fceed3 from peer 16938db5e56d4e518736bca3efe91dae at {username='slave'} at 127.0.0.1:50444: session id = 16938db5e56d4e518736bca3efe91dae-28185f37b35d46b7b1a4d37110fceed3
I20251023 20:18:04.889673 22070 tablet_copy_source_session.cc:215] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:18:04.890782 22306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28185f37b35d46b7b1a4d37110fceed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:04.895598 22306 tablet_copy_client.cc:806] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: tablet copy: Starting download of 0 data blocks...
I20251023 20:18:04.895808 22306 tablet_copy_client.cc:670] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: tablet copy: Starting download of 1 WAL segments...
I20251023 20:18:04.897101 22306 tablet_copy_client.cc:538] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:18:04.899690 22306 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: Bootstrap starting.
I20251023 20:18:04.904564 22311 ts_tablet_manager.cc:933] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: Initiating tablet copy from peer a4e6557426d044a1929898c88af7be09 (127.15.81.193:46685)
I20251023 20:18:04.905416 22311 tablet_copy_client.cc:323] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.193:46685
I20251023 20:18:04.905906 22070 tablet_copy_service.cc:140] P a4e6557426d044a1929898c88af7be09: Received BeginTabletCopySession request for tablet 8321320ad7664bb68aaafce2f1aa341e from peer 16938db5e56d4e518736bca3efe91dae ({username='slave'} at 127.0.0.1:50444)
I20251023 20:18:04.906026 22070 tablet_copy_service.cc:161] P a4e6557426d044a1929898c88af7be09: Beginning new tablet copy session on tablet 8321320ad7664bb68aaafce2f1aa341e from peer 16938db5e56d4e518736bca3efe91dae at {username='slave'} at 127.0.0.1:50444: session id = 16938db5e56d4e518736bca3efe91dae-8321320ad7664bb68aaafce2f1aa341e
I20251023 20:18:04.906085 22306 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:04.906324 22306 tablet_bootstrap.cc:492] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: Bootstrap complete.
I20251023 20:18:04.906512 22306 ts_tablet_manager.cc:1403] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.006s
I20251023 20:18:04.907183 22306 raft_consensus.cc:359] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.907317 22306 raft_consensus.cc:740] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 16938db5e56d4e518736bca3efe91dae, State: Initialized, Role: LEARNER
I20251023 20:18:04.907425 22306 consensus_queue.cc:260] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } attrs { replace: true } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.908227 22070 tablet_copy_source_session.cc:215] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:18:04.908558 22306 ts_tablet_manager.cc:1434] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 20:18:04.909310 22314 ts_tablet_manager.cc:933] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: Initiating tablet copy from peer 8a617d97848847c79d0559f2680f6ef8 (127.15.81.194:46425)
I20251023 20:18:04.910159 22070 tablet_copy_service.cc:342] P a4e6557426d044a1929898c88af7be09: Request end of tablet copy session 16938db5e56d4e518736bca3efe91dae-28185f37b35d46b7b1a4d37110fceed3 received from {username='slave'} at 127.0.0.1:50444
I20251023 20:18:04.910248 22070 tablet_copy_service.cc:434] P a4e6557426d044a1929898c88af7be09: ending tablet copy session 16938db5e56d4e518736bca3efe91dae-28185f37b35d46b7b1a4d37110fceed3 on tablet 28185f37b35d46b7b1a4d37110fceed3 with peer 16938db5e56d4e518736bca3efe91dae
I20251023 20:18:04.910532 22311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8321320ad7664bb68aaafce2f1aa341e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:04.911103 22314 tablet_copy_client.cc:323] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:46425
I20251023 20:18:04.916368 22311 tablet_copy_client.cc:806] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: tablet copy: Starting download of 0 data blocks...
I20251023 20:18:04.916631 22311 tablet_copy_client.cc:670] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: tablet copy: Starting download of 1 WAL segments...
I20251023 20:18:04.918097 22311 tablet_copy_client.cc:538] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:18:04.919662 22150 tablet_copy_service.cc:140] P 8a617d97848847c79d0559f2680f6ef8: Received BeginTabletCopySession request for tablet 1f10a9e0110747188abf9e526a6e816c from peer 16938db5e56d4e518736bca3efe91dae ({username='slave'} at 127.0.0.1:38746)
I20251023 20:18:04.919796 22150 tablet_copy_service.cc:161] P 8a617d97848847c79d0559f2680f6ef8: Beginning new tablet copy session on tablet 1f10a9e0110747188abf9e526a6e816c from peer 16938db5e56d4e518736bca3efe91dae at {username='slave'} at 127.0.0.1:38746: session id = 16938db5e56d4e518736bca3efe91dae-1f10a9e0110747188abf9e526a6e816c
I20251023 20:18:04.921238 22311 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: Bootstrap starting.
I20251023 20:18:04.922520 22150 tablet_copy_source_session.cc:215] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:18:04.923489 22314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f10a9e0110747188abf9e526a6e816c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:04.927050 22311 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:04.927332 22311 tablet_bootstrap.cc:492] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: Bootstrap complete.
I20251023 20:18:04.927541 22311 ts_tablet_manager.cc:1403] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251023 20:18:04.928208 22311 raft_consensus.cc:359] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.928395 22314 tablet_copy_client.cc:806] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: tablet copy: Starting download of 0 data blocks...
I20251023 20:18:04.928469 22311 raft_consensus.cc:740] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 16938db5e56d4e518736bca3efe91dae, State: Initialized, Role: LEARNER
I20251023 20:18:04.928597 22314 tablet_copy_client.cc:670] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: tablet copy: Starting download of 1 WAL segments...
I20251023 20:18:04.928665 22311 consensus_queue.cc:260] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } attrs { replace: true } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.929167 22311 ts_tablet_manager.cc:1434] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251023 20:18:04.929744 22070 tablet_copy_service.cc:342] P a4e6557426d044a1929898c88af7be09: Request end of tablet copy session 16938db5e56d4e518736bca3efe91dae-8321320ad7664bb68aaafce2f1aa341e received from {username='slave'} at 127.0.0.1:50444
I20251023 20:18:04.929839 22070 tablet_copy_service.cc:434] P a4e6557426d044a1929898c88af7be09: ending tablet copy session 16938db5e56d4e518736bca3efe91dae-8321320ad7664bb68aaafce2f1aa341e on tablet 8321320ad7664bb68aaafce2f1aa341e with peer 16938db5e56d4e518736bca3efe91dae
I20251023 20:18:04.930050 22314 tablet_copy_client.cc:538] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:18:04.932940 22314 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: Bootstrap starting.
I20251023 20:18:04.938150 22314 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:04.938519 22314 tablet_bootstrap.cc:492] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: Bootstrap complete.
I20251023 20:18:04.938732 22314 ts_tablet_manager.cc:1403] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20251023 20:18:04.939435 22314 raft_consensus.cc:359] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.939638 22314 raft_consensus.cc:740] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 16938db5e56d4e518736bca3efe91dae, State: Initialized, Role: LEARNER
I20251023 20:18:04.939806 22314 consensus_queue.cc:260] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a4e6557426d044a1929898c88af7be09" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 46685 } attrs { replace: true } } peers { permanent_uuid: "8a617d97848847c79d0559f2680f6ef8" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 46425 } } peers { permanent_uuid: "09b892db91c241458ecde269e5b26587" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 46071 } } peers { permanent_uuid: "16938db5e56d4e518736bca3efe91dae" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 33685 } attrs { promote: true } }
I20251023 20:18:04.940214 22314 ts_tablet_manager.cc:1434] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:18:04.940778 22150 tablet_copy_service.cc:342] P 8a617d97848847c79d0559f2680f6ef8: Request end of tablet copy session 16938db5e56d4e518736bca3efe91dae-1f10a9e0110747188abf9e526a6e816c received from {username='slave'} at 127.0.0.1:38746
I20251023 20:18:04.940903 22150 tablet_copy_service.cc:434] P 8a617d97848847c79d0559f2680f6ef8: ending tablet copy session 16938db5e56d4e518736bca3efe91dae-1f10a9e0110747188abf9e526a6e816c on tablet 1f10a9e0110747188abf9e526a6e816c with peer 16938db5e56d4e518736bca3efe91dae
I20251023 20:18:05.328380 21963 raft_consensus.cc:1217] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251023 20:18:05.400334 21963 raft_consensus.cc:1217] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251023 20:18:05.464356 21963 raft_consensus.cc:1217] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20251023 20:18:05.742410 15687 tablet_server.cc:178] TabletServer@127.15.81.193:46685 shutting down...
W20251023 20:18:05.746567 21632 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20251023 20:18:05.748140 21634 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:18:05.748215 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 20:18:05.750507 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:05.750752 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.750928 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 3 LEADER]: Raft consensus shutting down.
I20251023 20:18:05.751296 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Raft consensus is shut down!
W20251023 20:18:05.751652 21634 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:18:05.751708 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 20:18:05.751901 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.752029 15687 raft_consensus.cc:2243] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 3 LEADER]: Raft consensus shutting down.
I20251023 20:18:05.752286 15687 raft_consensus.cc:2272] T 2c765af5165d40eabfd0dec0ef96d4fd P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.752737 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.752907 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:18:05.753193 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P a4e6557426d044a1929898c88af7be09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.753633 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.753759 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.753952 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P a4e6557426d044a1929898c88af7be09 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.758615 15687 tablet_server.cc:195] TabletServer@127.15.81.193:46685 shutdown complete.
I20251023 20:18:05.762364 15687 tablet_server.cc:178] TabletServer@127.15.81.194:46425 shutting down...
W20251023 20:18:05.763617 21634 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251023 20:18:05.763708 21634 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251023 20:18:05.779572 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:05.779771 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.779911 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.780067 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.780481 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.780598 15687 raft_consensus.cc:2243] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.780714 15687 raft_consensus.cc:2272] T 2c765af5165d40eabfd0dec0ef96d4fd P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.781188 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.781313 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.781461 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P 8a617d97848847c79d0559f2680f6ef8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.781865 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.781983 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 LEADER]: Raft consensus shutting down.
I20251023 20:18:05.782300 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P 8a617d97848847c79d0559f2680f6ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.801755 15687 tablet_server.cc:195] TabletServer@127.15.81.194:46425 shutdown complete.
I20251023 20:18:05.805024 15687 tablet_server.cc:178] TabletServer@127.15.81.195:46071 shutting down...
I20251023 20:18:05.812330 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:05.812529 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.812682 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.812863 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.813344 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.813441 15687 raft_consensus.cc:2243] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.813553 15687 raft_consensus.cc:2272] T 2c765af5165d40eabfd0dec0ef96d4fd P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.813920 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.814011 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.814137 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P 09b892db91c241458ecde269e5b26587 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.814520 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:05.814599 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:05.814708 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P 09b892db91c241458ecde269e5b26587 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.828060 15687 tablet_server.cc:195] TabletServer@127.15.81.195:46071 shutdown complete.
I20251023 20:18:05.837857 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:18:05.845520 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:05.845727 15687 tablet_replica.cc:333] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae: stopping tablet replica
I20251023 20:18:05.845899 15687 raft_consensus.cc:2243] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Raft consensus shutting down.
I20251023 20:18:05.846042 15687 raft_consensus.cc:2272] T 1f10a9e0110747188abf9e526a6e816c P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Raft consensus is shut down!
I20251023 20:18:05.846557 15687 tablet_replica.cc:333] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae: stopping tablet replica
I20251023 20:18:05.846684 15687 raft_consensus.cc:2243] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Raft consensus shutting down.
I20251023 20:18:05.846802 15687 raft_consensus.cc:2272] T 28185f37b35d46b7b1a4d37110fceed3 P 16938db5e56d4e518736bca3efe91dae [term 3 LEARNER]: Raft consensus is shut down!
I20251023 20:18:05.847191 15687 tablet_replica.cc:333] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae: stopping tablet replica
I20251023 20:18:05.847303 15687 raft_consensus.cc:2243] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae [term 2 LEARNER]: Raft consensus shutting down.
I20251023 20:18:05.847412 15687 raft_consensus.cc:2272] T 8321320ad7664bb68aaafce2f1aa341e P 16938db5e56d4e518736bca3efe91dae [term 2 LEARNER]: Raft consensus is shut down!
I20251023 20:18:05.860591 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:18:05.863517 15687 master.cc:561] Master@127.15.81.254:37985 shutting down...
I20251023 20:18:05.870752 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:05.870919 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:05.871002 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P bef72f756d014c56bc85a84cbce29f66: stopping tablet replica
I20251023 20:18:05.883842 15687 master.cc:583] Master@127.15.81.254:37985 shutdown complete.
[ OK ] AutoRebalancerTest.TestHandlingFailedTservers (11749 ms)
[ RUN ] AutoRebalancerTest.TestDeletedTables
I20251023 20:18:05.900171 15687 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.81.254:41703
I20251023 20:18:05.900756 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:05.903182 22324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:05.903330 22325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:05.903884 22327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:05.903905 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:05.904223 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:05.904287 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:05.904335 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250685904331 us; error 0 us; skew 500 ppm
I20251023 20:18:05.904500 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:05.905517 15687 webserver.cc:492] Webserver started at http://127.15.81.254:34079/ using document root <none> and password file <none>
I20251023 20:18:05.905700 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:05.905755 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:18:05.905869 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:18:05.906422 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/master-0-root/instance:
uuid: "c8ea4045ecb74a829b54f4177fcb31cf"
format_stamp: "Formatted at 2025-10-23 20:18:05 on dist-test-slave-jr94"
I20251023 20:18:05.908742 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251023 20:18:05.910303 22332 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:05.910607 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:05.910692 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/master-0-root
uuid: "c8ea4045ecb74a829b54f4177fcb31cf"
format_stamp: "Formatted at 2025-10-23 20:18:05 on dist-test-slave-jr94"
I20251023 20:18:05.910782 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:05.916769 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:05.917183 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:05.926208 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.254:41703
I20251023 20:18:05.926235 22383 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.254:41703 every 8 connection(s)
I20251023 20:18:05.927316 22384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:05.931066 22384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Bootstrap starting.
I20251023 20:18:05.932065 22384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:05.933429 22384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: No bootstrap required, opened a new log
I20251023 20:18:05.933952 22384 raft_consensus.cc:359] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" member_type: VOTER }
I20251023 20:18:05.934065 22384 raft_consensus.cc:385] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:05.934118 22384 raft_consensus.cc:740] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ea4045ecb74a829b54f4177fcb31cf, State: Initialized, Role: FOLLOWER
I20251023 20:18:05.934303 22384 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" member_type: VOTER }
I20251023 20:18:05.934408 22384 raft_consensus.cc:399] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 20:18:05.934463 22384 raft_consensus.cc:493] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 20:18:05.934558 22384 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:05.936110 22384 raft_consensus.cc:515] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" member_type: VOTER }
I20251023 20:18:05.936266 22384 leader_election.cc:304] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8ea4045ecb74a829b54f4177fcb31cf; no voters:
I20251023 20:18:05.936600 22384 leader_election.cc:290] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20251023 20:18:05.936718 22388 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:05.937148 22388 raft_consensus.cc:697] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 1 LEADER]: Becoming Leader. State: Replica: c8ea4045ecb74a829b54f4177fcb31cf, State: Running, Role: LEADER
I20251023 20:18:05.937346 22388 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" member_type: VOTER }
I20251023 20:18:05.937399 22384 sys_catalog.cc:565] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [sys.catalog]: configured and running, proceeding with master startup.
I20251023 20:18:05.938086 22390 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8ea4045ecb74a829b54f4177fcb31cf. Latest consensus state: current_term: 1 leader_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" member_type: VOTER } }
I20251023 20:18:05.938222 22390 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [sys.catalog]: This master's current role is: LEADER
I20251023 20:18:05.938136 22389 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8ea4045ecb74a829b54f4177fcb31cf" member_type: VOTER } }
I20251023 20:18:05.938462 22389 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [sys.catalog]: This master's current role is: LEADER
I20251023 20:18:05.939399 22393 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 20:18:05.941329 22393 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 20:18:05.944096 15687 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 20:18:05.944238 22393 catalog_manager.cc:1357] Generated new cluster ID: 665d6ed0d45340c9a28d51d3459399fb
I20251023 20:18:05.944306 22393 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 20:18:05.954695 22393 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 20:18:05.955296 22393 catalog_manager.cc:1514] Loading token signing keys...
I20251023 20:18:05.963142 22393 catalog_manager.cc:6022] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Generated new TSK 0
I20251023 20:18:05.963318 22393 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 20:18:05.976560 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:05.978858 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:05.979111 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:05.979487 15687 server_base.cc:1047] running on GCE node
W20251023 20:18:05.979701 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:05.979948 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:05.980018 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:05.980047 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250685980044 us; error 0 us; skew 500 ppm
I20251023 20:18:05.980141 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:05.981247 15687 webserver.cc:492] Webserver started at http://127.15.81.193:44541/ using document root <none> and password file <none>
I20251023 20:18:05.981434 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:05.981485 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:18:05.981572 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:18:05.982090 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/instance:
uuid: "fa92bff5b38a4e3695970229546933d9"
format_stamp: "Formatted at 2025-10-23 20:18:05 on dist-test-slave-jr94"
I20251023 20:18:05.984347 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:18:05.985714 22414 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:05.986014 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251023 20:18:05.986088 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "fa92bff5b38a4e3695970229546933d9"
format_stamp: "Formatted at 2025-10-23 20:18:05 on dist-test-slave-jr94"
I20251023 20:18:05.986181 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:06.004082 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:06.004576 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:06.005039 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:06.005774 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:18:06.005820 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.005878 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:18:06.005921 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.016330 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:34071
I20251023 20:18:06.016374 22477 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:34071 every 8 connection(s)
I20251023 20:18:06.017783 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:06.020305 22482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:06.020834 22483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:06.021051 22485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:06.021489 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:06.021685 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:06.021737 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:06.021787 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250686021783 us; error 0 us; skew 500 ppm
I20251023 20:18:06.021914 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:06.023188 15687 webserver.cc:492] Webserver started at http://127.15.81.194:43975/ using document root <none> and password file <none>
I20251023 20:18:06.023394 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:06.023458 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:18:06.023562 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:18:06.024106 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/instance:
uuid: "024e5249825c42d0bac99a80f0b4f222"
format_stamp: "Formatted at 2025-10-23 20:18:06 on dist-test-slave-jr94"
I20251023 20:18:06.026189 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251023 20:18:06.027706 22490 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.028086 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:06.028086 22478 heartbeater.cc:344] Connected to a master server at 127.15.81.254:41703
I20251023 20:18:06.028260 22478 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:06.028209 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root
uuid: "024e5249825c42d0bac99a80f0b4f222"
format_stamp: "Formatted at 2025-10-23 20:18:06 on dist-test-slave-jr94"
I20251023 20:18:06.028322 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:06.028597 22478 heartbeater.cc:507] Master 127.15.81.254:41703 requested a full tablet report, sending...
I20251023 20:18:06.029421 22349 ts_manager.cc:194] Registered new tserver with Master: fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071)
I20251023 20:18:06.030290 22349 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37078
I20251023 20:18:06.044248 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:06.044679 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:06.045166 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:06.045840 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:18:06.045890 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.045948 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:18:06.045984 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.055496 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.194:33515
I20251023 20:18:06.055526 22552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.194:33515 every 8 connection(s)
I20251023 20:18:06.057034 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:06.059520 22556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:06.059844 22557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:06.060094 22559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:06.060688 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:06.060945 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:06.061046 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:06.061087 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250686061083 us; error 0 us; skew 500 ppm
I20251023 20:18:06.061221 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:06.062503 22553 heartbeater.cc:344] Connected to a master server at 127.15.81.254:41703
I20251023 20:18:06.062611 15687 webserver.cc:492] Webserver started at http://127.15.81.195:32775/ using document root <none> and password file <none>
I20251023 20:18:06.062655 22553 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:06.062801 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:06.062865 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:18:06.062968 22553 heartbeater.cc:507] Master 127.15.81.254:41703 requested a full tablet report, sending...
I20251023 20:18:06.062991 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:18:06.063738 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/instance:
uuid: "b512283c6beb499997235d58fa0b2eae"
format_stamp: "Formatted at 2025-10-23 20:18:06 on dist-test-slave-jr94"
I20251023 20:18:06.063781 22349 ts_manager.cc:194] Registered new tserver with Master: 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:06.064509 22349 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37090
I20251023 20:18:06.066273 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251023 20:18:06.067562 22564 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.067842 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:06.067914 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root
uuid: "b512283c6beb499997235d58fa0b2eae"
format_stamp: "Formatted at 2025-10-23 20:18:06 on dist-test-slave-jr94"
I20251023 20:18:06.068001 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:06.082696 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:06.083117 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:06.083585 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:06.084292 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:18:06.084347 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.084410 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:18:06.084457 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:06.094880 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.195:33227
I20251023 20:18:06.094906 22627 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.195:33227 every 8 connection(s)
I20251023 20:18:06.100348 22628 heartbeater.cc:344] Connected to a master server at 127.15.81.254:41703
I20251023 20:18:06.100476 22628 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:06.100756 22628 heartbeater.cc:507] Master 127.15.81.254:41703 requested a full tablet report, sending...
I20251023 20:18:06.101519 22349 ts_manager.cc:194] Registered new tserver with Master: b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:06.102178 22349 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37092
I20251023 20:18:06.102404 15687 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006741007s
I20251023 20:18:07.031704 22478 heartbeater.cc:499] Master 127.15.81.254:41703 was elected leader, sending a full tablet report...
I20251023 20:18:07.065860 22553 heartbeater.cc:499] Master 127.15.81.254:41703 was elected leader, sending a full tablet report...
I20251023 20:18:07.103442 22628 heartbeater.cc:499] Master 127.15.81.254:41703 was elected leader, sending a full tablet report...
I20251023 20:18:07.129657 15687 test_util.cc:276] Using random seed: 563157043
I20251023 20:18:07.142546 22349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37100:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:18:07.143231 22349 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:18:07.159883 22518 tablet_service.cc:1505] Processing CreateTablet for tablet 03ab8ce022984c379448ed4cd5b7ada6 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:18:07.160372 22518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ab8ce022984c379448ed4cd5b7ada6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.159981 22515 tablet_service.cc:1505] Processing CreateTablet for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:18:07.164103 22515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acf1d8c7934480fbf2873b9e9fa77d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.165095 22591 tablet_service.cc:1505] Processing CreateTablet for tablet ecc57d78878149a7b4ceb2a1ca320aab (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:18:07.165463 22591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc57d78878149a7b4ceb2a1ca320aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.165095 22593 tablet_service.cc:1505] Processing CreateTablet for tablet 03ab8ce022984c379448ed4cd5b7ada6 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:18:07.166338 22593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ab8ce022984c379448ed4cd5b7ada6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.169342 22516 tablet_service.cc:1505] Processing CreateTablet for tablet ecc57d78878149a7b4ceb2a1ca320aab (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:18:07.169798 22516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc57d78878149a7b4ceb2a1ca320aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.170188 22590 tablet_service.cc:1505] Processing CreateTablet for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:18:07.170531 22590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acf1d8c7934480fbf2873b9e9fa77d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.173259 22517 tablet_service.cc:1505] Processing CreateTablet for tablet 513ec58123084ca6903177be676f9bf9 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:18:07.173740 22517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513ec58123084ca6903177be676f9bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.177028 22592 tablet_service.cc:1505] Processing CreateTablet for tablet 513ec58123084ca6903177be676f9bf9 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:18:07.177526 22592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513ec58123084ca6903177be676f9bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.180456 22648 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.182433 22648 tablet_bootstrap.cc:654] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.191058 22650 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.192632 22650 tablet_bootstrap.cc:654] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.193782 22648 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.193987 22648 ts_tablet_manager.cc:1403] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20251023 20:18:07.194782 22650 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.194752 22648 raft_consensus.cc:359] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.194911 22650 ts_tablet_manager.cc:1403] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:18:07.194938 22648 raft_consensus.cc:385] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.195047 22648 raft_consensus.cc:740] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.195276 22648 consensus_queue.cc:260] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.195580 22650 raft_consensus.cc:359] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.195950 22648 ts_tablet_manager.cc:1434] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:18:07.195796 22650 raft_consensus.cc:385] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.196161 22650 raft_consensus.cc:740] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.196225 22648 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.196453 22650 consensus_queue.cc:260] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.197604 22648 tablet_bootstrap.cc:654] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.199777 22442 tablet_service.cc:1505] Processing CreateTablet for tablet 03ab8ce022984c379448ed4cd5b7ada6 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:18:07.200177 22442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ab8ce022984c379448ed4cd5b7ada6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.200541 22439 tablet_service.cc:1505] Processing CreateTablet for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:18:07.200987 22439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acf1d8c7934480fbf2873b9e9fa77d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.201388 22648 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.201483 22648 ts_tablet_manager.cc:1403] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251023 20:18:07.201920 22648 raft_consensus.cc:359] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.202034 22648 raft_consensus.cc:385] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.202066 22648 raft_consensus.cc:740] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.202237 22648 consensus_queue.cc:260] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.202653 22648 ts_tablet_manager.cc:1434] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:18:07.203786 22440 tablet_service.cc:1505] Processing CreateTablet for tablet ecc57d78878149a7b4ceb2a1ca320aab (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:18:07.204164 22440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc57d78878149a7b4ceb2a1ca320aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.204309 22648 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.205192 22441 tablet_service.cc:1505] Processing CreateTablet for tablet 513ec58123084ca6903177be676f9bf9 (DEFAULT_TABLE table=test-workload [id=7c71496130c44c69a7f17a234ef4e548]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:18:07.205530 22441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513ec58123084ca6903177be676f9bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.206525 22655 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.206714 22648 tablet_bootstrap.cc:654] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.207937 22655 tablet_bootstrap.cc:654] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.208345 22648 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.208449 22648 ts_tablet_manager.cc:1403] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20251023 20:18:07.209147 22648 raft_consensus.cc:359] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.209272 22648 raft_consensus.cc:385] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.209354 22648 raft_consensus.cc:740] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.209534 22648 consensus_queue.cc:260] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.210317 22648 ts_tablet_manager.cc:1434] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:18:07.211279 22648 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.213884 22648 tablet_bootstrap.cc:654] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.214974 22650 ts_tablet_manager.cc:1434] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20251023 20:18:07.215214 22650 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.215282 22655 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.215466 22655 ts_tablet_manager.cc:1403] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20251023 20:18:07.215785 22648 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.215934 22648 ts_tablet_manager.cc:1403] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251023 20:18:07.216485 22650 tablet_bootstrap.cc:654] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.216616 22648 raft_consensus.cc:359] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.216828 22648 raft_consensus.cc:385] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.216961 22648 raft_consensus.cc:740] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.217087 22655 raft_consensus.cc:359] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.217347 22655 raft_consensus.cc:385] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.217485 22655 raft_consensus.cc:740] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.217749 22655 consensus_queue.cc:260] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.217267 22648 consensus_queue.cc:260] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.218429 22650 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.218591 22650 ts_tablet_manager.cc:1403] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20251023 20:18:07.219229 22650 raft_consensus.cc:359] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.219439 22650 raft_consensus.cc:385] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.219548 22650 raft_consensus.cc:740] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.218469 22648 ts_tablet_manager.cc:1434] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:18:07.219736 22650 consensus_queue.cc:260] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.220209 22655 ts_tablet_manager.cc:1434] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251023 20:18:07.220628 22655 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.220624 22650 ts_tablet_manager.cc:1434] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251023 20:18:07.221086 22650 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.221879 22655 tablet_bootstrap.cc:654] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.223309 22650 tablet_bootstrap.cc:654] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.224596 22655 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.224713 22655 ts_tablet_manager.cc:1403] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251023 20:18:07.224745 22650 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.224843 22650 ts_tablet_manager.cc:1403] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251023 20:18:07.225319 22655 raft_consensus.cc:359] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.225339 22650 raft_consensus.cc:359] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.225513 22655 raft_consensus.cc:385] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.225584 22650 raft_consensus.cc:385] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.225668 22650 raft_consensus.cc:740] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.225600 22655 raft_consensus.cc:740] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.225840 22650 consensus_queue.cc:260] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.226107 22655 consensus_queue.cc:260] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.226276 22650 ts_tablet_manager.cc:1434] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251023 20:18:07.226599 22655 ts_tablet_manager.cc:1434] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:18:07.226835 22650 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.226835 22655 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.228188 22655 tablet_bootstrap.cc:654] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.228188 22650 tablet_bootstrap.cc:654] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.230247 22655 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.230358 22655 ts_tablet_manager.cc:1403] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251023 20:18:07.230657 22650 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.230758 22650 ts_tablet_manager.cc:1403] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251023 20:18:07.230885 22655 raft_consensus.cc:359] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.231021 22655 raft_consensus.cc:385] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.231123 22655 raft_consensus.cc:740] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.231263 22650 raft_consensus.cc:359] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.231377 22650 raft_consensus.cc:385] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.231422 22650 raft_consensus.cc:740] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.231330 22655 consensus_queue.cc:260] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.231626 22650 consensus_queue.cc:260] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.231722 22655 ts_tablet_manager.cc:1434] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251023 20:18:07.231971 22655 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.232028 22650 ts_tablet_manager.cc:1434] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:18:07.233181 22655 tablet_bootstrap.cc:654] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.234683 22655 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.234791 22655 ts_tablet_manager.cc:1403] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:18:07.235401 22655 raft_consensus.cc:359] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.235517 22655 raft_consensus.cc:385] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.235601 22655 raft_consensus.cc:740] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.235783 22655 consensus_queue.cc:260] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.236157 22655 ts_tablet_manager.cc:1434] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251023 20:18:07.275300 22653 raft_consensus.cc:493] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.275445 22653 raft_consensus.cc:515] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.276036 22653 leader_election.cc:290] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.281831 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ecc57d78878149a7b4ceb2a1ca320aab" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:07.281836 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ecc57d78878149a7b4ceb2a1ca320aab" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:07.282020 22603 raft_consensus.cc:2468] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 0.
I20251023 20:18:07.282091 22453 raft_consensus.cc:2468] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 0.
I20251023 20:18:07.282426 22492 leader_election.cc:304] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae; no voters:
I20251023 20:18:07.282666 22653 raft_consensus.cc:2804] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.282732 22653 raft_consensus.cc:493] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.282778 22653 raft_consensus.cc:3060] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.284987 22653 raft_consensus.cc:515] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.285518 22653 leader_election.cc:290] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.285778 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ecc57d78878149a7b4ceb2a1ca320aab" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:07.285853 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ecc57d78878149a7b4ceb2a1ca320aab" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:07.285914 22453 raft_consensus.cc:3060] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.285986 22603 raft_consensus.cc:3060] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.287909 22453 raft_consensus.cc:2468] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:07.288007 22603 raft_consensus.cc:2468] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:07.288230 22491 leader_election.cc:304] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.288440 22653 raft_consensus.cc:2804] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.288777 22653 raft_consensus.cc:697] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Becoming Leader. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Running, Role: LEADER
I20251023 20:18:07.289034 22653 consensus_queue.cc:237] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.291241 22347 catalog_manager.cc:5649] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: term changed from 0 to 1, leader changed from <none> to 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:07.294011 22660 raft_consensus.cc:493] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.294116 22660 raft_consensus.cc:515] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.294782 22660 leader_election.cc:290] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.300062 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:07.300074 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:07.300215 22603 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 0.
I20251023 20:18:07.300261 22528 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 0.
I20251023 20:18:07.300539 22416 leader_election.cc:304] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.300755 22660 raft_consensus.cc:2804] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.300827 22660 raft_consensus.cc:493] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.300925 22660 raft_consensus.cc:3060] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.303009 22660 raft_consensus.cc:515] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.303635 22660 leader_election.cc:290] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 election: Requested vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.303913 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:07.304056 22528 raft_consensus.cc:3060] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.304117 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:07.304271 22603 raft_consensus.cc:3060] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.306064 22528 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 1.
I20251023 20:18:07.306201 22603 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 1.
I20251023 20:18:07.306418 22418 leader_election.cc:304] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.306644 22660 raft_consensus.cc:2804] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.306979 22660 raft_consensus.cc:697] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: Becoming Leader. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Running, Role: LEADER
I20251023 20:18:07.307201 22660 consensus_queue.cc:237] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.309373 22347 catalog_manager.cc:5649] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa92bff5b38a4e3695970229546933d9 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "fa92bff5b38a4e3695970229546933d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:07.343257 22653 raft_consensus.cc:493] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.343494 22653 raft_consensus.cc:515] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.344123 22653 leader_election.cc:290] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.344570 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4acf1d8c7934480fbf2873b9e9fa77d2" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:07.344585 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4acf1d8c7934480fbf2873b9e9fa77d2" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:07.344736 22453 raft_consensus.cc:2468] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 0.
I20251023 20:18:07.344823 22603 raft_consensus.cc:2468] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 0.
I20251023 20:18:07.345106 22491 leader_election.cc:304] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.345305 22653 raft_consensus.cc:2804] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.345391 22653 raft_consensus.cc:493] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.345451 22653 raft_consensus.cc:3060] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.347143 22653 raft_consensus.cc:515] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.347574 22653 leader_election.cc:290] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.347889 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4acf1d8c7934480fbf2873b9e9fa77d2" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:07.347930 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4acf1d8c7934480fbf2873b9e9fa77d2" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:07.348038 22453 raft_consensus.cc:3060] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.348062 22603 raft_consensus.cc:3060] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.349926 22603 raft_consensus.cc:2468] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:07.350054 22453 raft_consensus.cc:2468] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:07.350211 22492 leader_election.cc:304] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae; no voters:
I20251023 20:18:07.350620 22653 raft_consensus.cc:2804] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.350759 22653 raft_consensus.cc:697] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Becoming Leader. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Running, Role: LEADER
I20251023 20:18:07.351002 22653 consensus_queue.cc:237] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.353101 22347 catalog_manager.cc:5649] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: term changed from 0 to 1, leader changed from <none> to 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:07.415385 22660 raft_consensus.cc:493] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.415601 22660 raft_consensus.cc:515] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.416154 22660 leader_election.cc:290] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.416579 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:07.416550 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:07.416807 22603 raft_consensus.cc:2468] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 0.
I20251023 20:18:07.416805 22528 raft_consensus.cc:2468] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 0.
I20251023 20:18:07.417213 22416 leader_election.cc:304] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.417465 22660 raft_consensus.cc:2804] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.417544 22660 raft_consensus.cc:493] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.417627 22660 raft_consensus.cc:3060] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.419446 22660 raft_consensus.cc:515] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.419986 22660 leader_election.cc:290] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 election: Requested vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.420342 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:07.420418 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:07.420493 22528 raft_consensus.cc:3060] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.420614 22603 raft_consensus.cc:3060] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.422439 22528 raft_consensus.cc:2468] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 1.
I20251023 20:18:07.422518 22603 raft_consensus.cc:2468] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 1.
I20251023 20:18:07.422770 22418 leader_election.cc:304] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.423048 22660 raft_consensus.cc:2804] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.423166 22660 raft_consensus.cc:697] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: Becoming Leader. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Running, Role: LEADER
I20251023 20:18:07.423400 22660 consensus_queue.cc:237] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.425586 22347 catalog_manager.cc:5649] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa92bff5b38a4e3695970229546933d9 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "fa92bff5b38a4e3695970229546933d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:07.453339 15687 test_util.cc:276] Using random seed: 563480705
I20251023 20:18:07.463876 22347 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37112:
name: "dugtrio"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251023 20:18:07.464558 22347 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table dugtrio in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 20:18:07.471300 22440 tablet_service.cc:1505] Processing CreateTablet for tablet e521f2cd06b24cc8a4363a4869f2357a (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:18:07.471464 22515 tablet_service.cc:1505] Processing CreateTablet for tablet e521f2cd06b24cc8a4363a4869f2357a (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:18:07.471720 22440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e521f2cd06b24cc8a4363a4869f2357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.471765 22515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e521f2cd06b24cc8a4363a4869f2357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.472183 22439 tablet_service.cc:1505] Processing CreateTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:18:07.472605 22439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e141cc0f7e9a45ee96c67ee6a083793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.473555 22590 tablet_service.cc:1505] Processing CreateTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:18:07.473969 22590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e141cc0f7e9a45ee96c67ee6a083793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.479053 22592 tablet_service.cc:1505] Processing CreateTablet for tablet e521f2cd06b24cc8a4363a4869f2357a (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251023 20:18:07.479420 22592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e521f2cd06b24cc8a4363a4869f2357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.479537 22518 tablet_service.cc:1505] Processing CreateTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251023 20:18:07.479864 22518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e141cc0f7e9a45ee96c67ee6a083793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.480120 22650 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.481127 22516 tablet_service.cc:1505] Processing CreateTablet for tablet 8b705a5eafb249678837c6b48b12ffb2 (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:18:07.481472 22516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b705a5eafb249678837c6b48b12ffb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.481813 22650 tablet_bootstrap.cc:654] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.485108 22517 tablet_service.cc:1505] Processing CreateTablet for tablet 3296f59295e142cfa155d9a54c8e0abf (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:18:07.485471 22517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3296f59295e142cfa155d9a54c8e0abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.486624 22648 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.486900 22655 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.488209 22655 tablet_bootstrap.cc:654] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.488236 22648 tablet_bootstrap.cc:654] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.486897 22650 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.489250 22650 ts_tablet_manager.cc:1403] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20251023 20:18:07.489598 22593 tablet_service.cc:1505] Processing CreateTablet for tablet 8b705a5eafb249678837c6b48b12ffb2 (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:18:07.490094 22655 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.490058 22650 raft_consensus.cc:359] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.490235 22655 ts_tablet_manager.cc:1403] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20251023 20:18:07.490285 22650 raft_consensus.cc:385] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.490363 22650 raft_consensus.cc:740] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.490901 22655 raft_consensus.cc:359] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.490842 22439 tablet_service.cc:1505] Processing CreateTablet for tablet 8b705a5eafb249678837c6b48b12ffb2 (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251023 20:18:07.491086 22655 raft_consensus.cc:385] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.490056 22593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b705a5eafb249678837c6b48b12ffb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.491158 22655 raft_consensus.cc:740] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.491432 22439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b705a5eafb249678837c6b48b12ffb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.491465 22655 consensus_queue.cc:260] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.491932 22655 ts_tablet_manager.cc:1434] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251023 20:18:07.492209 22655 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.493824 22655 tablet_bootstrap.cc:654] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.496345 22650 consensus_queue.cc:260] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.497058 22650 ts_tablet_manager.cc:1434] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20251023 20:18:07.497372 22650 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.497866 22592 tablet_service.cc:1505] Processing CreateTablet for tablet 3296f59295e142cfa155d9a54c8e0abf (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:18:07.498289 22592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3296f59295e142cfa155d9a54c8e0abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.499439 22650 tablet_bootstrap.cc:654] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.506285 22440 tablet_service.cc:1505] Processing CreateTablet for tablet 3296f59295e142cfa155d9a54c8e0abf (DEFAULT_TABLE table=dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251023 20:18:07.508582 22440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3296f59295e142cfa155d9a54c8e0abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:07.513662 22655 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.513662 22648 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.513943 22655 ts_tablet_manager.cc:1403] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.004s
I20251023 20:18:07.513979 22648 ts_tablet_manager.cc:1403] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.011s
I20251023 20:18:07.514571 22655 raft_consensus.cc:359] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.514761 22655 raft_consensus.cc:385] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.514726 22648 raft_consensus.cc:359] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.514842 22655 raft_consensus.cc:740] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.514904 22648 raft_consensus.cc:385] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.514997 22648 raft_consensus.cc:740] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.515105 22655 consensus_queue.cc:260] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.515199 22648 consensus_queue.cc:260] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.515574 22655 ts_tablet_manager.cc:1434] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:18:07.515794 22650 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.515846 22655 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.515925 22650 ts_tablet_manager.cc:1403] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20251023 20:18:07.516464 22650 raft_consensus.cc:359] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.516615 22650 raft_consensus.cc:385] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.516716 22650 raft_consensus.cc:740] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.516942 22650 consensus_queue.cc:260] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.517354 22650 ts_tablet_manager.cc:1434] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:07.517690 22655 tablet_bootstrap.cc:654] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.518484 22650 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.518205 22648 ts_tablet_manager.cc:1434] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251023 20:18:07.519513 22648 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.520246 22650 tablet_bootstrap.cc:654] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.521198 22648 tablet_bootstrap.cc:654] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.526580 22655 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.526721 22655 ts_tablet_manager.cc:1403] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20251023 20:18:07.526791 22650 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.526937 22650 ts_tablet_manager.cc:1403] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20251023 20:18:07.527180 22648 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.527320 22648 ts_tablet_manager.cc:1403] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20251023 20:18:07.527534 22650 raft_consensus.cc:359] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.527681 22650 raft_consensus.cc:385] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.527796 22650 raft_consensus.cc:740] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.528028 22650 consensus_queue.cc:260] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.527544 22655 raft_consensus.cc:359] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.528278 22655 raft_consensus.cc:385] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.528357 22655 raft_consensus.cc:740] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.528429 22650 ts_tablet_manager.cc:1434] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:18:07.528556 22655 consensus_queue.cc:260] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.528771 22650 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Bootstrap starting.
I20251023 20:18:07.528061 22648 raft_consensus.cc:359] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.529049 22648 raft_consensus.cc:385] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.529181 22648 raft_consensus.cc:740] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.529462 22648 consensus_queue.cc:260] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.530474 22650 tablet_bootstrap.cc:654] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.531177 22655 ts_tablet_manager.cc:1434] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251023 20:18:07.532589 22648 ts_tablet_manager.cc:1434] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251023 20:18:07.533371 22648 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.533424 22655 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:07.535143 22655 tablet_bootstrap.cc:654] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.535295 22648 tablet_bootstrap.cc:654] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.537246 22655 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: No bootstrap required, opened a new log
I20251023 20:18:07.537442 22655 ts_tablet_manager.cc:1403] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251023 20:18:07.538129 22650 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: No bootstrap required, opened a new log
I20251023 20:18:07.538328 22650 ts_tablet_manager.cc:1403] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20251023 20:18:07.538393 22655 raft_consensus.cc:359] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.538682 22655 raft_consensus.cc:385] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.538841 22655 raft_consensus.cc:740] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.539068 22650 raft_consensus.cc:359] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.539211 22650 raft_consensus.cc:385] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.539299 22650 raft_consensus.cc:740] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.539187 22655 consensus_queue.cc:260] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.539506 22650 consensus_queue.cc:260] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.539984 22650 ts_tablet_manager.cc:1434] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251023 20:18:07.540742 22655 ts_tablet_manager.cc:1434] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:18:07.543519 22648 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.543658 22648 ts_tablet_manager.cc:1403] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20251023 20:18:07.544164 22648 raft_consensus.cc:359] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.544302 22648 raft_consensus.cc:385] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.544385 22648 raft_consensus.cc:740] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.544570 22648 consensus_queue.cc:260] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.544978 22648 ts_tablet_manager.cc:1434] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251023 20:18:07.545261 22648 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Bootstrap starting.
I20251023 20:18:07.546679 22648 tablet_bootstrap.cc:654] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Neither blocks nor log segments found. Creating new log.
I20251023 20:18:07.548311 22648 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: No bootstrap required, opened a new log
I20251023 20:18:07.548431 22648 ts_tablet_manager.cc:1403] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251023 20:18:07.548969 22648 raft_consensus.cc:359] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.549124 22648 raft_consensus.cc:385] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 20:18:07.549197 22648 raft_consensus.cc:740] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Initialized, Role: FOLLOWER
I20251023 20:18:07.549377 22648 consensus_queue.cc:260] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.549717 22648 ts_tablet_manager.cc:1434] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:07.567852 22653 raft_consensus.cc:493] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.567977 22653 raft_consensus.cc:515] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.568444 22653 leader_election.cc:290] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.568807 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e521f2cd06b24cc8a4363a4869f2357a" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:07.568872 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e521f2cd06b24cc8a4363a4869f2357a" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:07.569082 22453 raft_consensus.cc:2468] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 0.
I20251023 20:18:07.569094 22603 raft_consensus.cc:2468] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 0.
I20251023 20:18:07.569470 22491 leader_election.cc:304] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.569721 22653 raft_consensus.cc:2804] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.569823 22653 raft_consensus.cc:493] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.569918 22653 raft_consensus.cc:3060] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.571659 22653 raft_consensus.cc:515] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.572224 22653 leader_election.cc:290] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.572542 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e521f2cd06b24cc8a4363a4869f2357a" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:07.572592 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e521f2cd06b24cc8a4363a4869f2357a" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:07.572732 22453 raft_consensus.cc:3060] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.572734 22603 raft_consensus.cc:3060] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.575456 22453 raft_consensus.cc:2468] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:07.575456 22603 raft_consensus.cc:2468] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:07.575901 22492 leader_election.cc:304] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.576153 22653 raft_consensus.cc:2804] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.576282 22653 raft_consensus.cc:697] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Becoming Leader. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Running, Role: LEADER
I20251023 20:18:07.576504 22653 consensus_queue.cc:237] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.578986 22349 catalog_manager.cc:5649] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: term changed from 0 to 1, leader changed from <none> to 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194). New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:07.580650 22652 raft_consensus.cc:493] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.580787 22652 raft_consensus.cc:515] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.581633 22652 leader_election.cc:290] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:07.587966 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e141cc0f7e9a45ee96c67ee6a083793b" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:07.588160 22528 raft_consensus.cc:2468] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 0.
I20251023 20:18:07.588186 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e141cc0f7e9a45ee96c67ee6a083793b" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:07.588371 22453 raft_consensus.cc:2468] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 0.
I20251023 20:18:07.588493 22568 leader_election.cc:304] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae; no voters:
I20251023 20:18:07.588754 22652 raft_consensus.cc:2804] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.588877 22652 raft_consensus.cc:493] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.588954 22652 raft_consensus.cc:3060] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.590672 22652 raft_consensus.cc:515] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.591123 22652 leader_election.cc:290] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:07.591444 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e141cc0f7e9a45ee96c67ee6a083793b" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:07.591537 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e141cc0f7e9a45ee96c67ee6a083793b" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:07.591629 22453 raft_consensus.cc:3060] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.591675 22528 raft_consensus.cc:3060] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.594195 22453 raft_consensus.cc:2468] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 1.
I20251023 20:18:07.594201 22528 raft_consensus.cc:2468] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 1.
I20251023 20:18:07.594630 22568 leader_election.cc:304] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.594887 22652 raft_consensus.cc:2804] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.595223 22652 raft_consensus.cc:697] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 1 LEADER]: Becoming Leader. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Running, Role: LEADER
I20251023 20:18:07.595475 22652 consensus_queue.cc:237] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.597450 22349 catalog_manager.cc:5649] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae reported cstate change: term changed from 0 to 1, leader changed from <none> to b512283c6beb499997235d58fa0b2eae (127.15.81.195). New cstate: current_term: 1 leader_uuid: "b512283c6beb499997235d58fa0b2eae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } }
I20251023 20:18:07.598981 22652 raft_consensus.cc:493] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.599156 22652 raft_consensus.cc:515] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.599733 22652 leader_election.cc:290] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:07.599993 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3296f59295e142cfa155d9a54c8e0abf" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:07.600183 22453 raft_consensus.cc:2468] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 0.
I20251023 20:18:07.600447 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3296f59295e142cfa155d9a54c8e0abf" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:07.600556 22565 leader_election.cc:304] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.600653 22528 raft_consensus.cc:2468] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 0.
I20251023 20:18:07.600904 22652 raft_consensus.cc:2804] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.601079 22652 raft_consensus.cc:493] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.601190 22652 raft_consensus.cc:3060] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.602864 22652 raft_consensus.cc:515] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.603368 22652 leader_election.cc:290] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:07.603642 22453 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3296f59295e142cfa155d9a54c8e0abf" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:07.603731 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3296f59295e142cfa155d9a54c8e0abf" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:07.603809 22453 raft_consensus.cc:3060] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.603942 22528 raft_consensus.cc:3060] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.605588 22453 raft_consensus.cc:2468] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 1.
I20251023 20:18:07.605792 22528 raft_consensus.cc:2468] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 1.
I20251023 20:18:07.605916 22565 leader_election.cc:304] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.606248 22652 raft_consensus.cc:2804] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.606400 22652 raft_consensus.cc:697] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 1 LEADER]: Becoming Leader. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Running, Role: LEADER
I20251023 20:18:07.606616 22652 consensus_queue.cc:237] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.608635 22349 catalog_manager.cc:5649] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae reported cstate change: term changed from 0 to 1, leader changed from <none> to b512283c6beb499997235d58fa0b2eae (127.15.81.195). New cstate: current_term: 1 leader_uuid: "b512283c6beb499997235d58fa0b2eae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } }
I20251023 20:18:07.738112 22653 consensus_queue.cc:1048] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.742753 22653 consensus_queue.cc:1048] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.828433 22653 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.832584 22668 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.884379 22660 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.888054 22672 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.891953 22672 raft_consensus.cc:493] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 20:18:07.892129 22672 raft_consensus.cc:515] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.892683 22672 leader_election.cc:290] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.893061 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b705a5eafb249678837c6b48b12ffb2" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:07.893201 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b705a5eafb249678837c6b48b12ffb2" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:07.893290 22528 raft_consensus.cc:2468] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 0.
I20251023 20:18:07.893414 22603 raft_consensus.cc:2468] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 0.
I20251023 20:18:07.893880 22418 leader_election.cc:304] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.894253 22672 raft_consensus.cc:2804] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 20:18:07.894330 22672 raft_consensus.cc:493] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 20:18:07.894407 22672 raft_consensus.cc:3060] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.896111 22672 raft_consensus.cc:515] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.896679 22672 leader_election.cc:290] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 election: Requested vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:07.896962 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b705a5eafb249678837c6b48b12ffb2" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:07.897122 22528 raft_consensus.cc:3060] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.897610 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b705a5eafb249678837c6b48b12ffb2" candidate_uuid: "fa92bff5b38a4e3695970229546933d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:07.897797 22603 raft_consensus.cc:3060] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 0 FOLLOWER]: Advancing to term 1
I20251023 20:18:07.899688 22603 raft_consensus.cc:2468] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 1.
I20251023 20:18:07.899816 22528 raft_consensus.cc:2468] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa92bff5b38a4e3695970229546933d9 in term 1.
I20251023 20:18:07.900127 22416 leader_election.cc:304] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters:
I20251023 20:18:07.900398 22672 raft_consensus.cc:2804] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 20:18:07.900568 22672 raft_consensus.cc:697] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: Becoming Leader. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Running, Role: LEADER
I20251023 20:18:07.900799 22672 consensus_queue.cc:237] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:07.907140 22348 catalog_manager.cc:5649] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa92bff5b38a4e3695970229546933d9 (127.15.81.193). New cstate: current_term: 1 leader_uuid: "fa92bff5b38a4e3695970229546933d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:07.934340 22672 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:07.937919 15687 tablet_server.cc:178] TabletServer@127.15.81.193:0 shutting down...
I20251023 20:18:07.942334 22672 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251023 20:18:07.952013 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:07.952677 15687 tablet_replica.cc:333] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.952935 15687 raft_consensus.cc:2243] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:07.953184 15687 pending_rounds.cc:70] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Trying to abort 1 pending ops.
I20251023 20:18:07.953294 15687 pending_rounds.cc:77] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082817644142592 op_type: NO_OP noop_request { }
I20251023 20:18:07.953423 15687 raft_consensus.cc:2889] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:18:07.953589 15687 raft_consensus.cc:2272] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.954493 15687 tablet_replica.cc:333] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.954672 15687 raft_consensus.cc:2243] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:07.954840 15687 raft_consensus.cc:2272] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.955475 15687 tablet_replica.cc:333] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.955591 15687 raft_consensus.cc:2243] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:07.955714 15687 raft_consensus.cc:2272] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.956215 15687 tablet_replica.cc:333] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.956350 15687 raft_consensus.cc:2243] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:07.956558 15687 raft_consensus.cc:2272] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.957154 15687 tablet_replica.cc:333] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.957262 15687 raft_consensus.cc:2243] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:07.957386 15687 raft_consensus.cc:2272] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.957872 15687 tablet_replica.cc:333] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.957990 15687 raft_consensus.cc:2243] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:07.958096 15687 raft_consensus.cc:2272] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.958539 15687 tablet_replica.cc:333] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.958643 15687 raft_consensus.cc:2243] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:07.958859 15687 raft_consensus.cc:2272] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.959335 15687 tablet_replica.cc:333] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: stopping tablet replica
I20251023 20:18:07.959453 15687 raft_consensus.cc:2243] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:07.959554 15687 raft_consensus.cc:2272] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:07.964522 15687 tablet_server.cc:195] TabletServer@127.15.81.193:0 shutdown complete.
W20251023 20:18:07.975004 22491 consensus_peers.cc:597] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 -> Peer fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): Couldn't send request to peer fa92bff5b38a4e3695970229546933d9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:34071: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:07.984500 22652 consensus_queue.cc:1048] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:18:07.985419 22565 consensus_peers.cc:597] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae -> Peer fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): Couldn't send request to peer fa92bff5b38a4e3695970229546933d9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:34071: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 20:18:07.989044 22565 consensus_peers.cc:597] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae -> Peer fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): Couldn't send request to peer fa92bff5b38a4e3695970229546933d9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:34071: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:08.043330 22668 consensus_queue.cc:1048] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 20:18:08.211683 22652 consensus_queue.cc:1048] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251023 20:18:08.223839 22491 consensus_peers.cc:597] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 -> Peer fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): Couldn't send request to peer fa92bff5b38a4e3695970229546933d9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:34071: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251023 20:18:08.290392 22491 consensus_peers.cc:597] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 -> Peer fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): Couldn't send request to peer fa92bff5b38a4e3695970229546933d9. Status: Network error: Client connection negotiation failed: client connection to 127.15.81.193:34071: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:08.904585 22716 consensus_queue.cc:579] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [LEADER]: Leader has been unable to successfully communicate with peer fa92bff5b38a4e3695970229546933d9 for more than 1 seconds (1.298s)
I20251023 20:18:08.913933 22653 consensus_queue.cc:579] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Leader has been unable to successfully communicate with peer fa92bff5b38a4e3695970229546933d9 for more than 1 seconds (1.337s)
W20251023 20:18:08.947512 22398 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver fa92bff5b38a4e3695970229546933d9 not available for placement
I20251023 20:18:08.982259 22349 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37100:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251023 20:18:08.982426 22349 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37100:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251023 20:18:08.983955 22716 consensus_queue.cc:579] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [LEADER]: Leader has been unable to successfully communicate with peer fa92bff5b38a4e3695970229546933d9 for more than 1 seconds (1.388s)
I20251023 20:18:08.987573 22349 catalog_manager.cc:5936] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Sending DeleteTablet for 3 replicas of tablet e521f2cd06b24cc8a4363a4869f2357a
I20251023 20:18:08.988361 22592 tablet_service.cc:1552] Processing DeleteTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:50404
I20251023 20:18:08.988394 22517 tablet_service.cc:1552] Processing DeleteTablet for tablet e521f2cd06b24cc8a4363a4869f2357a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:37808
I20251023 20:18:08.988945 22730 tablet_replica.cc:333] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:08.989151 22730 raft_consensus.cc:2243] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:08.989437 22730 raft_consensus.cc:2272] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251023 20:18:08.989804 22333 catalog_manager.cc:4712] TS fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e521f2cd06b24cc8a4363a4869f2357a: Network error: Client connection negotiation failed: client connection to 127.15.81.193:34071: connect: Connection refused (error 111)
I20251023 20:18:08.989923 22349 catalog_manager.cc:5936] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Sending DeleteTablet for 3 replicas of tablet e141cc0f7e9a45ee96c67ee6a083793b
I20251023 20:18:08.990096 22592 tablet_service.cc:1552] Processing DeleteTablet for tablet e521f2cd06b24cc8a4363a4869f2357a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:50404
I20251023 20:18:08.990374 22730 ts_tablet_manager.cc:1916] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:08.990793 22592 tablet_service.cc:1552] Processing DeleteTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:50404
I20251023 20:18:08.990799 22517 tablet_service.cc:1552] Processing DeleteTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:37808
I20251023 20:18:08.991016 22349 catalog_manager.cc:5936] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Sending DeleteTablet for 3 replicas of tablet 3296f59295e142cfa155d9a54c8e0abf
I20251023 20:18:08.991510 22349 catalog_manager.cc:5936] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: Sending DeleteTablet for 3 replicas of tablet 8b705a5eafb249678837c6b48b12ffb2
I20251023 20:18:08.991647 22517 tablet_service.cc:1552] Processing DeleteTablet for tablet 3296f59295e142cfa155d9a54c8e0abf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:37808
I20251023 20:18:08.991688 22592 tablet_service.cc:1552] Processing DeleteTablet for tablet 3296f59295e142cfa155d9a54c8e0abf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:50404
I20251023 20:18:08.992331 22517 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b705a5eafb249678837c6b48b12ffb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:37808
I20251023 20:18:08.992373 22592 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b705a5eafb249678837c6b48b12ffb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:50404
I20251023 20:18:08.989849 22729 tablet_replica.cc:333] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:08.993312 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 20:18:08.993500 22729 raft_consensus.cc:2243] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:08.993796 22729 raft_consensus.cc:2272] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:08.994546 22729 ts_tablet_manager.cc:1916] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251023 20:18:08.995949 22732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:08.996505 22733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:08.997707 22730 ts_tablet_manager.cc:1929] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:08.997900 22730 log.cc:1199] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/e521f2cd06b24cc8a4363a4869f2357a
I20251023 20:18:08.998417 22730 ts_tablet_manager.cc:1950] T e521f2cd06b24cc8a4363a4869f2357a P 024e5249825c42d0bac99a80f0b4f222: Deleting consensus metadata
I20251023 20:18:08.999716 22730 tablet_replica.cc:333] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:08.999811 22336 catalog_manager.cc:4985] TS 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515): tablet e521f2cd06b24cc8a4363a4869f2357a (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:08.999940 22730 raft_consensus.cc:2243] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.000021 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:09.000104 22730 raft_consensus.cc:2272] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.000734 22730 ts_tablet_manager.cc:1916] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251023 20:18:09.000885 22735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:09.001276 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:09.001346 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:09.001376 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250689001373 us; error 0 us; skew 500 ppm
I20251023 20:18:09.001536 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:09.001652 22729 ts_tablet_manager.cc:1929] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:09.001761 22729 log.cc:1199] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/e141cc0f7e9a45ee96c67ee6a083793b
I20251023 20:18:09.002164 22729 ts_tablet_manager.cc:1950] T e141cc0f7e9a45ee96c67ee6a083793b P b512283c6beb499997235d58fa0b2eae: Deleting consensus metadata
I20251023 20:18:09.002975 22334 catalog_manager.cc:4985] TS b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227): tablet e141cc0f7e9a45ee96c67ee6a083793b (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.003671 22729 tablet_replica.cc:333] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:09.003825 22729 raft_consensus.cc:2243] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.003953 22729 raft_consensus.cc:2272] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.004601 22729 ts_tablet_manager.cc:1916] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.005620 15687 webserver.cc:492] Webserver started at http://127.15.81.196:45883/ using document root <none> and password file <none>
I20251023 20:18:09.005833 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:09.005893 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 20:18:09.006002 15687 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 20:18:09.006639 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-3-root/instance:
uuid: "65552aac62fc40d49cefd020b8325ce8"
format_stamp: "Formatted at 2025-10-23 20:18:09 on dist-test-slave-jr94"
I20251023 20:18:09.007894 22730 ts_tablet_manager.cc:1929] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:09.008030 22730 log.cc:1199] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/e141cc0f7e9a45ee96c67ee6a083793b
I20251023 20:18:09.008348 22730 ts_tablet_manager.cc:1950] T e141cc0f7e9a45ee96c67ee6a083793b P 024e5249825c42d0bac99a80f0b4f222: Deleting consensus metadata
I20251023 20:18:09.008917 22729 ts_tablet_manager.cc:1929] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:09.009034 22729 log.cc:1199] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/e521f2cd06b24cc8a4363a4869f2357a
I20251023 20:18:09.009204 22336 catalog_manager.cc:4985] TS 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515): tablet e141cc0f7e9a45ee96c67ee6a083793b (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.009362 22729 ts_tablet_manager.cc:1950] T e521f2cd06b24cc8a4363a4869f2357a P b512283c6beb499997235d58fa0b2eae: Deleting consensus metadata
I20251023 20:18:09.009526 22730 tablet_replica.cc:333] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:09.009670 22730 raft_consensus.cc:2243] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.009742 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251023 20:18:09.009804 22730 raft_consensus.cc:2272] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.010190 22334 catalog_manager.cc:4985] TS b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227): tablet e521f2cd06b24cc8a4363a4869f2357a (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.010344 22730 ts_tablet_manager.cc:1916] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.010429 22729 tablet_replica.cc:333] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
W20251023 20:18:09.010453 22334 catalog_manager.cc:4952] TS b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227): delete failed for tablet e141cc0f7e9a45ee96c67ee6a083793b because the tablet was not found. No further retry: Not found: Tablet not found: e141cc0f7e9a45ee96c67ee6a083793b
I20251023 20:18:09.010561 22729 raft_consensus.cc:2243] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:09.010783 22729 raft_consensus.cc:2272] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.011268 22729 ts_tablet_manager.cc:1916] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.011466 22740 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:09.011834 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:09.011988 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-3-root
uuid: "65552aac62fc40d49cefd020b8325ce8"
format_stamp: "Formatted at 2025-10-23 20:18:09 on dist-test-slave-jr94"
I20251023 20:18:09.012639 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:09.015720 22729 ts_tablet_manager.cc:1929] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:09.015807 22729 log.cc:1199] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/3296f59295e142cfa155d9a54c8e0abf
I20251023 20:18:09.016196 22729 ts_tablet_manager.cc:1950] T 3296f59295e142cfa155d9a54c8e0abf P b512283c6beb499997235d58fa0b2eae: Deleting consensus metadata
I20251023 20:18:09.016693 22730 ts_tablet_manager.cc:1929] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:09.016827 22729 tablet_replica.cc:333] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:09.016902 22730 log.cc:1199] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/3296f59295e142cfa155d9a54c8e0abf
I20251023 20:18:09.016942 22334 catalog_manager.cc:4985] TS b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227): tablet 3296f59295e142cfa155d9a54c8e0abf (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.017076 22729 raft_consensus.cc:2243] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.017194 22729 raft_consensus.cc:2272] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.017417 22730 ts_tablet_manager.cc:1950] T 3296f59295e142cfa155d9a54c8e0abf P 024e5249825c42d0bac99a80f0b4f222: Deleting consensus metadata
I20251023 20:18:09.017638 22729 ts_tablet_manager.cc:1916] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.018371 22730 tablet_replica.cc:333] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:09.018486 22336 catalog_manager.cc:4985] TS 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515): tablet 3296f59295e142cfa155d9a54c8e0abf (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.018664 22730 raft_consensus.cc:2243] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.018829 22730 raft_consensus.cc:2272] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.019490 22730 ts_tablet_manager.cc:1916] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.021863 22729 ts_tablet_manager.cc:1929] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 20:18:09.021950 22729 log.cc:1199] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/8b705a5eafb249678837c6b48b12ffb2
I20251023 20:18:09.022243 22729 ts_tablet_manager.cc:1950] T 8b705a5eafb249678837c6b48b12ffb2 P b512283c6beb499997235d58fa0b2eae: Deleting consensus metadata
I20251023 20:18:09.023160 22334 catalog_manager.cc:4985] TS b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227): tablet 8b705a5eafb249678837c6b48b12ffb2 (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.025477 22730 ts_tablet_manager.cc:1929] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 20:18:09.025588 22730 log.cc:1199] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/8b705a5eafb249678837c6b48b12ffb2
I20251023 20:18:09.025868 22730 ts_tablet_manager.cc:1950] T 8b705a5eafb249678837c6b48b12ffb2 P 024e5249825c42d0bac99a80f0b4f222: Deleting consensus metadata
I20251023 20:18:09.026578 22336 catalog_manager.cc:4985] TS 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515): tablet 8b705a5eafb249678837c6b48b12ffb2 (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.035027 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:09.035452 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:09.035902 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:09.036628 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 20:18:09.036675 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:09.036757 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 20:18:09.036796 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:09.048357 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.196:37331
I20251023 20:18:09.048362 22802 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.196:37331 every 8 connection(s)
I20251023 20:18:09.050032 15687 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 20:18:09.052502 22807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 20:18:09.052589 22806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:09.055729 22803 heartbeater.cc:344] Connected to a master server at 127.15.81.254:41703
I20251023 20:18:09.055827 22803 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:09.056111 22803 heartbeater.cc:507] Master 127.15.81.254:41703 requested a full tablet report, sending...
I20251023 20:18:09.056828 22349 ts_manager.cc:194] Registered new tserver with Master: 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331)
I20251023 20:18:09.057768 22349 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37134
W20251023 20:18:09.058108 22809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 20:18:09.058204 15687 server_base.cc:1047] running on GCE node
I20251023 20:18:09.058630 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 20:18:09.058701 15687 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 20:18:09.058737 15687 hybrid_clock.cc:648] HybridClock initialized: now 1761250689058734 us; error 0 us; skew 500 ppm
I20251023 20:18:09.058883 15687 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 20:18:09.059939 15687 webserver.cc:492] Webserver started at http://127.15.81.193:44541/ using document root <none> and password file <none>
I20251023 20:18:09.060112 15687 fs_manager.cc:362] Metadata directory not provided
I20251023 20:18:09.060154 15687 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 20:18:09.062188 15687 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20251023 20:18:09.063758 22814 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251023 20:18:09.064100 15687 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:09.064178 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root
uuid: "fa92bff5b38a4e3695970229546933d9"
format_stamp: "Formatted at 2025-10-23 20:18:05 on dist-test-slave-jr94"
I20251023 20:18:09.064298 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 20:18:09.081462 15687 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251023 20:18:09.081873 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 20:18:09.082360 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 20:18:09.083423 22821 ts_tablet_manager.cc:548] Loading tablet metadata (0/8 complete)
I20251023 20:18:09.098843 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (8 total tablets, 8 live tablets)
I20251023 20:18:09.098934 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20251023 20:18:09.099031 15687 ts_tablet_manager.cc:600] Registering tablets (0/8 complete)
I20251023 20:18:09.100999 22821 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.104621 22821 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 20:18:09.105011 22821 tablet_bootstrap.cc:492] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.105234 22821 ts_tablet_manager.cc:1403] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20251023 20:18:09.105798 22821 raft_consensus.cc:359] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.105967 22821 raft_consensus.cc:740] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.106168 22821 consensus_queue.cc:260] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.106822 22821 ts_tablet_manager.cc:1434] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251023 20:18:09.107028 22821 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.109967 22821 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.110215 22821 tablet_bootstrap.cc:492] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.110406 22821 ts_tablet_manager.cc:1403] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20251023 20:18:09.110863 22821 raft_consensus.cc:359] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.110981 22821 raft_consensus.cc:740] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.111155 22821 consensus_queue.cc:260] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.111567 22821 ts_tablet_manager.cc:1434] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:18:09.111761 22821 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.112490 15687 ts_tablet_manager.cc:616] Registered 8 tablets
I20251023 20:18:09.112558 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.014s user 0.010s sys 0.000s
I20251023 20:18:09.114722 22821 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.115024 22821 tablet_bootstrap.cc:492] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.115285 22821 ts_tablet_manager.cc:1403] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251023 20:18:09.115878 22821 raft_consensus.cc:359] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.116051 22821 raft_consensus.cc:740] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.116253 22821 consensus_queue.cc:260] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.116613 22821 ts_tablet_manager.cc:1434] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251023 20:18:09.116786 22821 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.120441 22821 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.120750 22821 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.120994 22821 ts_tablet_manager.cc:1403] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251023 20:18:09.121510 22821 raft_consensus.cc:359] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.121630 22821 raft_consensus.cc:740] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.121809 22821 consensus_queue.cc:260] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.122159 22821 ts_tablet_manager.cc:1434] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:09.122361 22821 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.125490 22821 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.125761 22821 tablet_bootstrap.cc:492] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.125978 22821 ts_tablet_manager.cc:1403] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251023 20:18:09.126444 22821 raft_consensus.cc:359] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.126559 22821 raft_consensus.cc:740] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.126760 22821 consensus_queue.cc:260] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.127136 22821 ts_tablet_manager.cc:1434] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:09.127337 22821 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.131186 22821 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.131539 22821 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.131791 22821 ts_tablet_manager.cc:1403] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251023 20:18:09.132236 22821 raft_consensus.cc:359] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.132349 22821 raft_consensus.cc:740] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.132506 22821 consensus_queue.cc:260] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.132898 22821 ts_tablet_manager.cc:1434] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20251023 20:18:09.133101 22821 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.136595 22821 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.137044 22821 tablet_bootstrap.cc:492] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.137241 22821 ts_tablet_manager.cc:1403] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20251023 20:18:09.137696 22821 raft_consensus.cc:359] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.137831 22821 raft_consensus.cc:740] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.138058 22821 consensus_queue.cc:260] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.138473 22821 ts_tablet_manager.cc:1434] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:09.138700 22821 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Bootstrap starting.
I20251023 20:18:09.142481 22821 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:09.142859 22821 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Bootstrap complete.
I20251023 20:18:09.143157 22821 ts_tablet_manager.cc:1403] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251023 20:18:09.143678 15687 rpc_server.cc:307] RPC server started. Bound to: 127.15.81.193:34071
I20251023 20:18:09.143707 22887 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.81.193:34071 every 8 connection(s)
I20251023 20:18:09.143724 22821 raft_consensus.cc:359] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.143878 22821 raft_consensus.cc:740] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa92bff5b38a4e3695970229546933d9, State: Initialized, Role: FOLLOWER
I20251023 20:18:09.144074 22821 consensus_queue.cc:260] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.144448 22821 ts_tablet_manager.cc:1434] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251023 20:18:09.150197 22888 heartbeater.cc:344] Connected to a master server at 127.15.81.254:41703
I20251023 20:18:09.150305 22888 heartbeater.cc:461] Registering TS with master...
I20251023 20:18:09.150563 22888 heartbeater.cc:507] Master 127.15.81.254:41703 requested a full tablet report, sending...
I20251023 20:18:09.151791 22349 ts_manager.cc:194] Re-registered known tserver with Master: fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071)
I20251023 20:18:09.154278 22349 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37146
I20251023 20:18:09.155331 22888 heartbeater.cc:499] Master 127.15.81.254:41703 was elected leader, sending a full tablet report...
I20251023 20:18:09.174618 22848 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b705a5eafb249678837c6b48b12ffb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:47812
I20251023 20:18:09.175135 22892 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:09.175323 22892 raft_consensus.cc:2243] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.175447 22892 pending_rounds.cc:70] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Trying to abort 1 pending ops.
I20251023 20:18:09.175493 22892 pending_rounds.cc:77] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7214082817644142592 op_type: NO_OP noop_request { }
I20251023 20:18:09.175583 22892 raft_consensus.cc:2889] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251023 20:18:09.175664 22892 raft_consensus.cc:2272] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.176254 22892 ts_tablet_manager.cc:1916] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.180265 22848 tablet_service.cc:1552] Processing DeleteTablet for tablet e141cc0f7e9a45ee96c67ee6a083793b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:47812
I20251023 20:18:09.180559 22892 ts_tablet_manager.cc:1929] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 20:18:09.180642 22892 log.cc:1199] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/8b705a5eafb249678837c6b48b12ffb2
I20251023 20:18:09.180949 22892 ts_tablet_manager.cc:1950] T 8b705a5eafb249678837c6b48b12ffb2 P fa92bff5b38a4e3695970229546933d9: Deleting consensus metadata
I20251023 20:18:09.181725 22892 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:09.181811 22333 catalog_manager.cc:4985] TS fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): tablet 8b705a5eafb249678837c6b48b12ffb2 (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.181886 22892 raft_consensus.cc:2243] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.182024 22892 raft_consensus.cc:2272] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.182538 22892 ts_tablet_manager.cc:1916] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.186251 22892 ts_tablet_manager.cc:1929] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 20:18:09.186424 22892 log.cc:1199] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/e141cc0f7e9a45ee96c67ee6a083793b
I20251023 20:18:09.186703 22892 ts_tablet_manager.cc:1950] T e141cc0f7e9a45ee96c67ee6a083793b P fa92bff5b38a4e3695970229546933d9: Deleting consensus metadata
I20251023 20:18:09.187489 22333 catalog_manager.cc:4985] TS fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): tablet e141cc0f7e9a45ee96c67ee6a083793b (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.281733 22848 tablet_service.cc:1552] Processing DeleteTablet for tablet 3296f59295e142cfa155d9a54c8e0abf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:47812
I20251023 20:18:09.282034 22892 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:09.282183 22892 raft_consensus.cc:2243] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.282343 22892 raft_consensus.cc:2272] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.282991 22892 ts_tablet_manager.cc:1916] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.284293 22701 consensus_queue.cc:579] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Leader has been unable to successfully communicate with peer fa92bff5b38a4e3695970229546933d9 for more than 1 seconds (1.539s)
I20251023 20:18:09.287796 22892 ts_tablet_manager.cc:1929] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 20:18:09.287922 22892 log.cc:1199] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/3296f59295e142cfa155d9a54c8e0abf
I20251023 20:18:09.287955 22528 consensus_queue.cc:237] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:09.288304 22892 ts_tablet_manager.cc:1950] T 3296f59295e142cfa155d9a54c8e0abf P fa92bff5b38a4e3695970229546933d9: Deleting consensus metadata
I20251023 20:18:09.289417 22333 catalog_manager.cc:4985] TS fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): tablet 3296f59295e142cfa155d9a54c8e0abf (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.291128 22859 raft_consensus.cc:1275] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:18:09.291352 22603 raft_consensus.cc:1275] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:18:09.291626 22653 consensus_queue.cc:1048] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.291836 22668 consensus_queue.cc:1048] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.295100 22701 raft_consensus.cc:2955] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.295660 22603 raft_consensus.cc:2955] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.295738 22859 raft_consensus.cc:2955] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.298633 22336 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ecc57d78878149a7b4ceb2a1ca320aab with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20251023 20:18:09.300124 22492 consensus_peers.cc:597] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 -> Peer 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): Couldn't send request to peer 65552aac62fc40d49cefd020b8325ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecc57d78878149a7b4ceb2a1ca320aab. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:09.300194 22349 catalog_manager.cc:5649] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae reported cstate change: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.304129 22528 consensus_queue.cc:237] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.305478 22603 raft_consensus.cc:1275] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:18:09.305588 22859 raft_consensus.cc:1275] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:18:09.305932 22653 consensus_queue.cc:1048] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:09.306694 22653 consensus_queue.cc:1048] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:09.307657 22668 raft_consensus.cc:2955] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } }
I20251023 20:18:09.308246 22603 raft_consensus.cc:2955] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } }
I20251023 20:18:09.308929 22859 raft_consensus.cc:2955] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } }
I20251023 20:18:09.309293 22848 tablet_service.cc:1552] Processing DeleteTablet for tablet e521f2cd06b24cc8a4363a4869f2357a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 20:18:08 UTC) from {username='slave'} at 127.0.0.1:47812
I20251023 20:18:09.309558 22892 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:09.309732 22892 raft_consensus.cc:2243] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:09.309873 22892 raft_consensus.cc:2272] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:09.310540 22892 ts_tablet_manager.cc:1916] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 20:18:09.312335 22336 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet ecc57d78878149a7b4ceb2a1ca320aab with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:18:09.312902 22347 catalog_manager.cc:5649] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } }
I20251023 20:18:09.319873 22892 ts_tablet_manager.cc:1929] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 20:18:09.319999 22892 log.cc:1199] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/e521f2cd06b24cc8a4363a4869f2357a
I20251023 20:18:09.320343 22892 ts_tablet_manager.cc:1950] T e521f2cd06b24cc8a4363a4869f2357a P fa92bff5b38a4e3695970229546933d9: Deleting consensus metadata
I20251023 20:18:09.321383 22333 catalog_manager.cc:4985] TS fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): tablet e521f2cd06b24cc8a4363a4869f2357a (table dugtrio [id=eaff54596c2d42dfaf212858db2ca97d]) successfully deleted
I20251023 20:18:09.321800 22768 tablet_service.cc:1552] Processing DeleteTablet for tablet ecc57d78878149a7b4ceb2a1ca320aab with delete_type TABLET_DATA_TOMBSTONED (TS 65552aac62fc40d49cefd020b8325ce8 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:50890
W20251023 20:18:09.322532 22334 catalog_manager.cc:4952] TS 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): delete failed for tablet ecc57d78878149a7b4ceb2a1ca320aab because the tablet was not found. No further retry: Not found: Tablet not found: ecc57d78878149a7b4ceb2a1ca320aab
I20251023 20:18:09.327903 22653 consensus_queue.cc:579] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Leader has been unable to successfully communicate with peer fa92bff5b38a4e3695970229546933d9 for more than 1 seconds (1.493s)
I20251023 20:18:09.331063 22528 consensus_queue.cc:237] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:09.332944 22859 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 20:18:09.333047 22603 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251023 20:18:09.333205 22492 consensus_peers.cc:597] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 -> Peer 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): Couldn't send request to peer 65552aac62fc40d49cefd020b8325ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4acf1d8c7934480fbf2873b9e9fa77d2. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:09.333475 22701 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.333755 22668 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.336120 22653 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.336647 22859 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.336685 22603 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.340595 22348 catalog_manager.cc:5649] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:09.341106 22336 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 20:18:09.345221 22528 consensus_queue.cc:237] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.346518 22603 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:18:09.346542 22859 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 20:18:09.346936 22653 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:09.347121 22668 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:09.348814 22701 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } }
I20251023 20:18:09.349480 22603 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } }
I20251023 20:18:09.349426 22859 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } }
I20251023 20:18:09.351714 22336 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:18:09.353327 22349 catalog_manager.cc:5649] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: config changed from index 2 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) evicted. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } }
I20251023 20:18:09.356071 22768 tablet_service.cc:1552] Processing DeleteTablet for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 with delete_type TABLET_DATA_TOMBSTONED (TS 65552aac62fc40d49cefd020b8325ce8 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:50890
W20251023 20:18:09.356906 22334 catalog_manager.cc:4952] TS 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): delete failed for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 because the tablet was not found. No further retry: Not found: Tablet not found: 4acf1d8c7934480fbf2873b9e9fa77d2
I20251023 20:18:09.388592 22716 raft_consensus.cc:493] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.388795 22716 raft_consensus.cc:515] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.389192 22701 raft_consensus.cc:493] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.389351 22701 raft_consensus.cc:515] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.389603 22716 leader_election.cc:290] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071)
I20251023 20:18:09.390125 22701 leader_election.cc:290] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:09.390369 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:09.390488 22859 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:09.390408 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:09.390640 22603 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:09.390668 22859 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 1.
I20251023 20:18:09.390772 22528 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 1.
I20251023 20:18:09.391070 22492 leader_election.cc:304] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae; no voters:
I20251023 20:18:09.391206 22568 leader_election.cc:304] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, b512283c6beb499997235d58fa0b2eae; no voters:
I20251023 20:18:09.391465 22716 raft_consensus.cc:2804] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:09.391515 22701 raft_consensus.cc:2804] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:09.391579 22716 raft_consensus.cc:493] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting leader election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.391670 22701 raft_consensus.cc:493] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.391700 22716 raft_consensus.cc:3060] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:09.391772 22701 raft_consensus.cc:3060] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:09.394013 22701 raft_consensus.cc:515] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.394284 22716 raft_consensus.cc:515] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.394662 22701 leader_election.cc:290] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 2 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:09.394893 22859 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:09.395092 22859 raft_consensus.cc:3060] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:09.395208 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:09.395466 22528 raft_consensus.cc:2393] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b512283c6beb499997235d58fa0b2eae in current term 2: Already voted for candidate 024e5249825c42d0bac99a80f0b4f222 in this term.
I20251023 20:18:09.395679 22716 leader_election.cc:290] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 election: Requested vote from peers 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515), fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071)
I20251023 20:18:09.395305 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae"
I20251023 20:18:09.396032 22603 raft_consensus.cc:2393] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 024e5249825c42d0bac99a80f0b4f222 in current term 2: Already voted for candidate b512283c6beb499997235d58fa0b2eae in this term.
I20251023 20:18:09.397320 22860 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:09.397400 22858 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03ab8ce022984c379448ed4cd5b7ada6" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:09.397560 22859 raft_consensus.cc:2468] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024e5249825c42d0bac99a80f0b4f222 in term 2.
I20251023 20:18:09.397965 22491 leader_election.cc:304] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9; no voters: b512283c6beb499997235d58fa0b2eae
I20251023 20:18:09.398085 22565 leader_election.cc:304] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b512283c6beb499997235d58fa0b2eae; no voters: 024e5249825c42d0bac99a80f0b4f222, fa92bff5b38a4e3695970229546933d9
I20251023 20:18:09.398317 22701 raft_consensus.cc:2804] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:18:09.398379 22716 raft_consensus.cc:2749] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251023 20:18:09.398483 22701 raft_consensus.cc:697] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 LEADER]: Becoming Leader. State: Replica: 024e5249825c42d0bac99a80f0b4f222, State: Running, Role: LEADER
I20251023 20:18:09.398756 22701 consensus_queue.cc:237] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.400902 22348 catalog_manager.cc:5649] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: term changed from 1 to 2, leader changed from fa92bff5b38a4e3695970229546933d9 (127.15.81.193) to 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194). New cstate: current_term: 2 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:09.438124 22716 raft_consensus.cc:493] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.438336 22716 raft_consensus.cc:515] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.438933 22716 leader_election.cc:290] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:09.439328 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222" is_pre_election: true
I20251023 20:18:09.439330 22859 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:09.439605 22859 raft_consensus.cc:2468] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 1.
I20251023 20:18:09.439901 22565 leader_election.cc:304] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters: 024e5249825c42d0bac99a80f0b4f222
I20251023 20:18:09.440153 22716 raft_consensus.cc:2804] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 20:18:09.440245 22716 raft_consensus.cc:493] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Starting leader election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.440358 22716 raft_consensus.cc:3060] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:09.442199 22716 raft_consensus.cc:515] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.442718 22716 leader_election.cc:290] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 election: Requested vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:09.443053 22859 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9"
I20251023 20:18:09.443086 22528 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "b512283c6beb499997235d58fa0b2eae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "024e5249825c42d0bac99a80f0b4f222"
I20251023 20:18:09.443221 22859 raft_consensus.cc:3060] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:09.444501 22701 raft_consensus.cc:493] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fa92bff5b38a4e3695970229546933d9)
I20251023 20:18:09.444664 22701 raft_consensus.cc:515] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.445225 22701 leader_election.cc:290] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071), b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:09.445380 22859 raft_consensus.cc:2468] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b512283c6beb499997235d58fa0b2eae in term 2.
I20251023 20:18:09.445580 22858 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa92bff5b38a4e3695970229546933d9" is_pre_election: true
I20251023 20:18:09.445772 22565 leader_election.cc:304] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9; no voters: 024e5249825c42d0bac99a80f0b4f222
I20251023 20:18:09.445854 22858 raft_consensus.cc:2393] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 024e5249825c42d0bac99a80f0b4f222 in current term 2: Already voted for candidate b512283c6beb499997235d58fa0b2eae in this term.
I20251023 20:18:09.445883 22603 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "513ec58123084ca6903177be676f9bf9" candidate_uuid: "024e5249825c42d0bac99a80f0b4f222" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b512283c6beb499997235d58fa0b2eae" is_pre_election: true
I20251023 20:18:09.446075 22716 raft_consensus.cc:2804] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Leader election won for term 2
I20251023 20:18:09.446256 22716 raft_consensus.cc:697] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 LEADER]: Becoming Leader. State: Replica: b512283c6beb499997235d58fa0b2eae, State: Running, Role: LEADER
I20251023 20:18:09.446489 22716 consensus_queue.cc:237] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } }
I20251023 20:18:09.447854 22492 leader_election.cc:304] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 024e5249825c42d0bac99a80f0b4f222; no voters: b512283c6beb499997235d58fa0b2eae, fa92bff5b38a4e3695970229546933d9
I20251023 20:18:09.448110 22701 raft_consensus.cc:3060] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Advancing to term 2
I20251023 20:18:09.448901 22348 catalog_manager.cc:5649] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae reported cstate change: term changed from 1 to 2, leader changed from fa92bff5b38a4e3695970229546933d9 (127.15.81.193) to b512283c6beb499997235d58fa0b2eae (127.15.81.195). New cstate: current_term: 2 leader_uuid: "b512283c6beb499997235d58fa0b2eae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } }
I20251023 20:18:09.462700 22701 raft_consensus.cc:2749] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251023 20:18:09.808736 22858 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:18:09.809219 22701 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.813385 22603 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:18:09.813882 22668 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.898917 22858 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:18:09.899423 22716 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 20:18:09.902947 22528 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 20:18:09.903353 22909 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251023 20:18:09.949594 22398 auto_rebalancer.cc:249] could not retrieve auto-rebalancing replica moves: Not found: table 7c71496130c44c69a7f17a234ef4e548: could not find any suitable replica to move from server 024e5249825c42d0bac99a80f0b4f222 to server 65552aac62fc40d49cefd020b8325ce8
I20251023 20:18:10.059463 22803 heartbeater.cc:499] Master 127.15.81.254:41703 was elected leader, sending a full tablet report...
I20251023 20:18:10.957677 22528 consensus_queue.cc:237] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:10.959690 22858 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:18:10.959786 22603 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20251023 20:18:10.959975 22492 consensus_peers.cc:597] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 -> Peer 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): Couldn't send request to peer 65552aac62fc40d49cefd020b8325ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03ab8ce022984c379448ed4cd5b7ada6. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:10.960155 22919 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:10.961051 22915 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:10.963610 22915 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.964114 22603 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.964267 22858 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.968098 22347 catalog_manager.cc:5649] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae reported cstate change: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New cstate: current_term: 2 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.970940 22528 consensus_queue.cc:237] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:10.972862 22603 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251023 20:18:10.972965 22492 consensus_peers.cc:597] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 -> Peer 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): Couldn't send request to peer 65552aac62fc40d49cefd020b8325ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4acf1d8c7934480fbf2873b9e9fa77d2. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:10.973268 22858 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251023 20:18:10.973276 22915 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:10.973765 22922 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:10.976537 22922 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.980096 22858 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.983793 22349 catalog_manager.cc:5649] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 reported cstate change: config changed from index 3 to 4, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.986575 22603 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.991628 22603 consensus_queue.cc:237] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:10.993647 22858 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:18:10.994282 22911 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:10.995442 22528 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 20:18:10.996191 22933 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 20:18:10.996932 22933 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.997526 22858 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:10.999517 22528 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
W20251023 20:18:10.999984 22566 consensus_peers.cc:597] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae -> Peer 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196:37331): Couldn't send request to peer 65552aac62fc40d49cefd020b8325ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 513ec58123084ca6903177be676f9bf9. This is attempt 1: this message will repeat every 5th retry.
I20251023 20:18:11.001709 22348 catalog_manager.cc:5649] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 reported cstate change: config changed from index -1 to 3, NON_VOTER 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) added. New cstate: current_term: 2 leader_uuid: "b512283c6beb499997235d58fa0b2eae" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } } }
I20251023 20:18:11.368963 22939 ts_tablet_manager.cc:933] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: Initiating tablet copy from peer b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227)
I20251023 20:18:11.369634 22939 tablet_copy_client.cc:323] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.195:33227
I20251023 20:18:11.375764 22613 tablet_copy_service.cc:140] P b512283c6beb499997235d58fa0b2eae: Received BeginTabletCopySession request for tablet 513ec58123084ca6903177be676f9bf9 from peer 65552aac62fc40d49cefd020b8325ce8 ({username='slave'} at 127.0.0.1:50446)
I20251023 20:18:11.375910 22613 tablet_copy_service.cc:161] P b512283c6beb499997235d58fa0b2eae: Beginning new tablet copy session on tablet 513ec58123084ca6903177be676f9bf9 from peer 65552aac62fc40d49cefd020b8325ce8 at {username='slave'} at 127.0.0.1:50446: session id = 65552aac62fc40d49cefd020b8325ce8-513ec58123084ca6903177be676f9bf9
I20251023 20:18:11.378083 22613 tablet_copy_source_session.cc:215] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:18:11.379055 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513ec58123084ca6903177be676f9bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:11.383528 22939 tablet_copy_client.cc:806] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Starting download of 0 data blocks...
I20251023 20:18:11.383701 22939 tablet_copy_client.cc:670] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Starting download of 1 WAL segments...
I20251023 20:18:11.385170 22939 tablet_copy_client.cc:538] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:18:11.387269 22939 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap starting.
I20251023 20:18:11.392763 22939 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:11.393172 22939 tablet_bootstrap.cc:492] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap complete.
I20251023 20:18:11.393394 22939 ts_tablet_manager.cc:1403] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.004s
I20251023 20:18:11.393961 22939 raft_consensus.cc:359] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:11.394166 22939 raft_consensus.cc:740] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 65552aac62fc40d49cefd020b8325ce8, State: Initialized, Role: LEARNER
I20251023 20:18:11.394399 22939 consensus_queue.cc:260] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:11.395191 22939 ts_tablet_manager.cc:1434] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251023 20:18:11.395780 22613 tablet_copy_service.cc:342] P b512283c6beb499997235d58fa0b2eae: Request end of tablet copy session 65552aac62fc40d49cefd020b8325ce8-513ec58123084ca6903177be676f9bf9 received from {username='slave'} at 127.0.0.1:50446
I20251023 20:18:11.395880 22613 tablet_copy_service.cc:434] P b512283c6beb499997235d58fa0b2eae: ending tablet copy session 65552aac62fc40d49cefd020b8325ce8-513ec58123084ca6903177be676f9bf9 on tablet 513ec58123084ca6903177be676f9bf9 with peer 65552aac62fc40d49cefd020b8325ce8
I20251023 20:18:11.526124 22939 ts_tablet_manager.cc:933] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: Initiating tablet copy from peer 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:11.526751 22939 tablet_copy_client.cc:323] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:33515
I20251023 20:18:11.532647 22538 tablet_copy_service.cc:140] P 024e5249825c42d0bac99a80f0b4f222: Received BeginTabletCopySession request for tablet 03ab8ce022984c379448ed4cd5b7ada6 from peer 65552aac62fc40d49cefd020b8325ce8 ({username='slave'} at 127.0.0.1:37838)
I20251023 20:18:11.532757 22538 tablet_copy_service.cc:161] P 024e5249825c42d0bac99a80f0b4f222: Beginning new tablet copy session on tablet 03ab8ce022984c379448ed4cd5b7ada6 from peer 65552aac62fc40d49cefd020b8325ce8 at {username='slave'} at 127.0.0.1:37838: session id = 65552aac62fc40d49cefd020b8325ce8-03ab8ce022984c379448ed4cd5b7ada6
I20251023 20:18:11.534782 22538 tablet_copy_source_session.cc:215] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:18:11.535882 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03ab8ce022984c379448ed4cd5b7ada6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:11.541466 22939 tablet_copy_client.cc:806] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Starting download of 0 data blocks...
I20251023 20:18:11.541702 22939 tablet_copy_client.cc:670] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Starting download of 1 WAL segments...
I20251023 20:18:11.543038 22939 tablet_copy_client.cc:538] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:18:11.546147 22939 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap starting.
I20251023 20:18:11.551941 22939 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:11.552297 22939 tablet_bootstrap.cc:492] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap complete.
I20251023 20:18:11.552512 22939 ts_tablet_manager.cc:1403] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251023 20:18:11.553186 22939 raft_consensus.cc:359] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:11.553385 22939 raft_consensus.cc:740] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 65552aac62fc40d49cefd020b8325ce8, State: Initialized, Role: LEARNER
I20251023 20:18:11.553560 22939 consensus_queue.cc:260] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:11.554091 22939 ts_tablet_manager.cc:1434] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251023 20:18:11.554661 22538 tablet_copy_service.cc:342] P 024e5249825c42d0bac99a80f0b4f222: Request end of tablet copy session 65552aac62fc40d49cefd020b8325ce8-03ab8ce022984c379448ed4cd5b7ada6 received from {username='slave'} at 127.0.0.1:37838
I20251023 20:18:11.554764 22538 tablet_copy_service.cc:434] P 024e5249825c42d0bac99a80f0b4f222: ending tablet copy session 65552aac62fc40d49cefd020b8325ce8-03ab8ce022984c379448ed4cd5b7ada6 on tablet 03ab8ce022984c379448ed4cd5b7ada6 with peer 65552aac62fc40d49cefd020b8325ce8
I20251023 20:18:11.557534 22939 ts_tablet_manager.cc:933] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: Initiating tablet copy from peer 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515)
I20251023 20:18:11.558279 22939 tablet_copy_client.cc:323] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Beginning tablet copy session from remote peer at address 127.15.81.194:33515
I20251023 20:18:11.558760 22538 tablet_copy_service.cc:140] P 024e5249825c42d0bac99a80f0b4f222: Received BeginTabletCopySession request for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 from peer 65552aac62fc40d49cefd020b8325ce8 ({username='slave'} at 127.0.0.1:37838)
I20251023 20:18:11.558857 22538 tablet_copy_service.cc:161] P 024e5249825c42d0bac99a80f0b4f222: Beginning new tablet copy session on tablet 4acf1d8c7934480fbf2873b9e9fa77d2 from peer 65552aac62fc40d49cefd020b8325ce8 at {username='slave'} at 127.0.0.1:37838: session id = 65552aac62fc40d49cefd020b8325ce8-4acf1d8c7934480fbf2873b9e9fa77d2
I20251023 20:18:11.560669 22538 tablet_copy_source_session.cc:215] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 20:18:11.561686 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4acf1d8c7934480fbf2873b9e9fa77d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 20:18:11.566426 22939 tablet_copy_client.cc:806] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Starting download of 0 data blocks...
I20251023 20:18:11.566609 22939 tablet_copy_client.cc:670] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Starting download of 1 WAL segments...
I20251023 20:18:11.567919 22939 tablet_copy_client.cc:538] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 20:18:11.570297 22939 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap starting.
I20251023 20:18:11.575313 22939 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 20:18:11.575630 22939 tablet_bootstrap.cc:492] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: Bootstrap complete.
I20251023 20:18:11.575866 22939 ts_tablet_manager.cc:1403] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20251023 20:18:11.576522 22939 raft_consensus.cc:359] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:11.576685 22939 raft_consensus.cc:740] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 65552aac62fc40d49cefd020b8325ce8, State: Initialized, Role: LEARNER
I20251023 20:18:11.576893 22939 consensus_queue.cc:260] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: NON_VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: true } }
I20251023 20:18:11.577431 22939 ts_tablet_manager.cc:1434] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251023 20:18:11.578105 22538 tablet_copy_service.cc:342] P 024e5249825c42d0bac99a80f0b4f222: Request end of tablet copy session 65552aac62fc40d49cefd020b8325ce8-4acf1d8c7934480fbf2873b9e9fa77d2 received from {username='slave'} at 127.0.0.1:37838
I20251023 20:18:11.578217 22538 tablet_copy_service.cc:434] P 024e5249825c42d0bac99a80f0b4f222: ending tablet copy session 65552aac62fc40d49cefd020b8325ce8-4acf1d8c7934480fbf2873b9e9fa77d2 on tablet 4acf1d8c7934480fbf2873b9e9fa77d2 with peer 65552aac62fc40d49cefd020b8325ce8
I20251023 20:18:11.808439 22778 raft_consensus.cc:1217] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251023 20:18:11.971717 22778 raft_consensus.cc:1217] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251023 20:18:11.974838 22778 raft_consensus.cc:1217] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251023 20:18:12.200160 22949 raft_consensus.cc:1064] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: attempting to promote NON_VOTER 65552aac62fc40d49cefd020b8325ce8 to VOTER
I20251023 20:18:12.200942 22949 consensus_queue.cc:237] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } }
I20251023 20:18:12.203204 22778 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:18:12.203316 22527 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:18:12.203644 22911 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:12.203661 22858 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:18:12.203975 22950 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:12.204226 22911 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:12.207867 22950 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.210753 22858 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.211618 22778 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.212296 22348 catalog_manager.cc:5649] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae reported cstate change: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "b512283c6beb499997235d58fa0b2eae" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251023 20:18:12.217010 22527 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.225574 22603 consensus_queue.cc:237] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } }
I20251023 20:18:12.226954 22778 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 20:18:12.227195 22858 raft_consensus.cc:1275] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer b512283c6beb499997235d58fa0b2eae: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 20:18:12.227551 22911 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.227957 22950 consensus_queue.cc:1048] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.229522 22911 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.230069 22778 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.230306 22858 raft_consensus.cc:2955] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.233114 22334 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 513ec58123084ca6903177be676f9bf9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:18:12.233994 22347 catalog_manager.cc:5649] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae reported cstate change: config changed from index 4 to 5, VOTER 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194) evicted. New cstate: current_term: 2 leader_uuid: "b512283c6beb499997235d58fa0b2eae" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:18:12.237361 22517 tablet_service.cc:1552] Processing DeleteTablet for tablet 513ec58123084ca6903177be676f9bf9 with delete_type TABLET_DATA_TOMBSTONED (TS 024e5249825c42d0bac99a80f0b4f222 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37808
I20251023 20:18:12.237987 22962 tablet_replica.cc:333] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:12.238353 22962 raft_consensus.cc:2243] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:12.238593 22962 raft_consensus.cc:2272] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:12.239403 22962 ts_tablet_manager.cc:1916] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:18:12.243970 22962 ts_tablet_manager.cc:1929] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251023 20:18:12.244073 22962 log.cc:1199] T 513ec58123084ca6903177be676f9bf9 P 024e5249825c42d0bac99a80f0b4f222: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-1-root/wals/513ec58123084ca6903177be676f9bf9
I20251023 20:18:12.244710 22336 catalog_manager.cc:4985] TS 024e5249825c42d0bac99a80f0b4f222 (127.15.81.194:33515): tablet 513ec58123084ca6903177be676f9bf9 (table test-workload [id=7c71496130c44c69a7f17a234ef4e548]) successfully deleted
I20251023 20:18:12.409822 22951 raft_consensus.cc:1064] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: attempting to promote NON_VOTER 65552aac62fc40d49cefd020b8325ce8 to VOTER
I20251023 20:18:12.410562 22951 consensus_queue.cc:237] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } }
I20251023 20:18:12.412621 22858 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 20:18:12.412647 22602 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 20:18:12.413226 22778 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 LEARNER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251023 20:18:12.413185 22952 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.413411 22919 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.413882 22919 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.416996 22858 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.416416 22951 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.420933 22349 catalog_manager.cc:5649] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 reported cstate change: config changed from index 4 to 5, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.422159 22602 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.429986 22778 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } attrs { replace: true } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.432420 22528 consensus_queue.cc:237] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } }
I20251023 20:18:12.434072 22858 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 20:18:12.434427 22777 raft_consensus.cc:1275] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251023 20:18:12.434746 22919 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 20:18:12.435282 22919 consensus_queue.cc:1048] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 20:18:12.437598 22919 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b512283c6beb499997235d58fa0b2eae (127.15.81.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.438248 22777 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b512283c6beb499997235d58fa0b2eae (127.15.81.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.438485 22858 raft_consensus.cc:2955] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b512283c6beb499997235d58fa0b2eae (127.15.81.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.441756 22336 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:18:12.442736 22346 catalog_manager.cc:5649] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 reported cstate change: config changed from index 5 to 6, VOTER b512283c6beb499997235d58fa0b2eae (127.15.81.195) evicted. New cstate: current_term: 1 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } } peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.445602 22592 tablet_service.cc:1552] Processing DeleteTablet for tablet 4acf1d8c7934480fbf2873b9e9fa77d2 with delete_type TABLET_DATA_TOMBSTONED (TS b512283c6beb499997235d58fa0b2eae not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50404
I20251023 20:18:12.446218 22968 tablet_replica.cc:333] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:12.446409 22968 raft_consensus.cc:2243] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:12.446635 22968 raft_consensus.cc:2272] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:12.447364 22968 ts_tablet_manager.cc:1916] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:18:12.452378 22968 ts_tablet_manager.cc:1929] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251023 20:18:12.452451 22968 log.cc:1199] T 4acf1d8c7934480fbf2873b9e9fa77d2 P b512283c6beb499997235d58fa0b2eae: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-2-root/wals/4acf1d8c7934480fbf2873b9e9fa77d2
I20251023 20:18:12.452987 22334 catalog_manager.cc:4985] TS b512283c6beb499997235d58fa0b2eae (127.15.81.195:33227): tablet 4acf1d8c7934480fbf2873b9e9fa77d2 (table test-workload [id=7c71496130c44c69a7f17a234ef4e548]) successfully deleted
I20251023 20:18:12.595811 22951 raft_consensus.cc:1064] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: attempting to promote NON_VOTER 65552aac62fc40d49cefd020b8325ce8 to VOTER
I20251023 20:18:12.596583 22951 consensus_queue.cc:237] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } }
I20251023 20:18:12.598598 22858 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:18:12.598598 22777 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 LEARNER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:18:12.599112 22602 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251023 20:18:12.599192 22952 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:12.599604 22919 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:12.599793 22952 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 20:18:12.602742 22919 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.603333 22858 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.603677 22603 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.608176 22349 catalog_manager.cc:5649] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:18:12.608572 22777 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 65552aac62fc40d49cefd020b8325ce8 (127.15.81.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "fa92bff5b38a4e3695970229546933d9" member_type: VOTER last_known_addr { host: "127.15.81.193" port: 34071 } attrs { replace: true } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.612442 22528 consensus_queue.cc:237] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } }
I20251023 20:18:12.614122 22603 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 20:18:12.614259 22777 raft_consensus.cc:1275] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Refusing update from remote peer 024e5249825c42d0bac99a80f0b4f222: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251023 20:18:12.614522 22951 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.614754 22952 consensus_queue.cc:1048] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 20:18:12.616674 22952 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fa92bff5b38a4e3695970229546933d9 (127.15.81.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.617591 22602 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fa92bff5b38a4e3695970229546933d9 (127.15.81.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.620321 22777 raft_consensus.cc:2955] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fa92bff5b38a4e3695970229546933d9 (127.15.81.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } } }
I20251023 20:18:12.620777 22336 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 03ab8ce022984c379448ed4cd5b7ada6 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 20:18:12.621052 22347 catalog_manager.cc:5649] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 reported cstate change: config changed from index 4 to 5, VOTER fa92bff5b38a4e3695970229546933d9 (127.15.81.193) evicted. New cstate: current_term: 2 leader_uuid: "024e5249825c42d0bac99a80f0b4f222" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "024e5249825c42d0bac99a80f0b4f222" member_type: VOTER last_known_addr { host: "127.15.81.194" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b512283c6beb499997235d58fa0b2eae" member_type: VOTER last_known_addr { host: "127.15.81.195" port: 33227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65552aac62fc40d49cefd020b8325ce8" member_type: VOTER last_known_addr { host: "127.15.81.196" port: 37331 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251023 20:18:12.624224 22848 tablet_service.cc:1552] Processing DeleteTablet for tablet 03ab8ce022984c379448ed4cd5b7ada6 with delete_type TABLET_DATA_TOMBSTONED (TS fa92bff5b38a4e3695970229546933d9 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47812
I20251023 20:18:12.624912 22973 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:12.625077 22973 raft_consensus.cc:2243] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:12.625340 22973 raft_consensus.cc:2272] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:12.625958 22973 ts_tablet_manager.cc:1916] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 20:18:12.629874 22973 ts_tablet_manager.cc:1929] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251023 20:18:12.629946 22973 log.cc:1199] T 03ab8ce022984c379448ed4cd5b7ada6 P fa92bff5b38a4e3695970229546933d9: Deleting WAL directory at /tmp/dist-test-taskdeGdTj/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761250506895969-15687-0/minicluster-data/ts-0-root/wals/03ab8ce022984c379448ed4cd5b7ada6
I20251023 20:18:12.630484 22333 catalog_manager.cc:4985] TS fa92bff5b38a4e3695970229546933d9 (127.15.81.193:34071): tablet 03ab8ce022984c379448ed4cd5b7ada6 (table test-workload [id=7c71496130c44c69a7f17a234ef4e548]) successfully deleted
I20251023 20:18:14.557860 15687 tablet_server.cc:178] TabletServer@127.15.81.193:34071 shutting down...
I20251023 20:18:14.573005 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:14.573275 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:14.573438 15687 raft_consensus.cc:2243] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.573626 15687 raft_consensus.cc:2272] T ecc57d78878149a7b4ceb2a1ca320aab P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.574163 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:14.574318 15687 raft_consensus.cc:2243] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.574509 15687 raft_consensus.cc:2272] T 513ec58123084ca6903177be676f9bf9 P fa92bff5b38a4e3695970229546933d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.574895 15687 tablet_replica.cc:333] stopping tablet replica
I20251023 20:18:14.574981 15687 raft_consensus.cc:2243] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.575198 15687 raft_consensus.cc:2272] T 4acf1d8c7934480fbf2873b9e9fa77d2 P fa92bff5b38a4e3695970229546933d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.588935 15687 tablet_server.cc:195] TabletServer@127.15.81.193:34071 shutdown complete.
I20251023 20:18:14.592990 15687 tablet_server.cc:178] TabletServer@127.15.81.194:0 shutting down...
I20251023 20:18:14.600577 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:14.600773 15687 tablet_replica.cc:333] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:14.600958 15687 raft_consensus.cc:2243] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:14.601181 15687 raft_consensus.cc:2272] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.601588 15687 tablet_replica.cc:333] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:14.601682 15687 raft_consensus.cc:2243] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:14.601863 15687 raft_consensus.cc:2272] T ecc57d78878149a7b4ceb2a1ca320aab P 024e5249825c42d0bac99a80f0b4f222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.602236 15687 tablet_replica.cc:333] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222: stopping tablet replica
I20251023 20:18:14.602324 15687 raft_consensus.cc:2243] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:18:14.602488 15687 raft_consensus.cc:2272] T 03ab8ce022984c379448ed4cd5b7ada6 P 024e5249825c42d0bac99a80f0b4f222 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.616070 15687 tablet_server.cc:195] TabletServer@127.15.81.194:0 shutdown complete.
I20251023 20:18:14.619874 15687 tablet_server.cc:178] TabletServer@127.15.81.195:0 shutting down...
W20251023 20:18:14.623958 22398 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver fa92bff5b38a4e3695970229546933d9 not available for placement
I20251023 20:18:14.630332 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:14.630517 15687 tablet_replica.cc:333] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:14.630704 15687 raft_consensus.cc:2243] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.630911 15687 raft_consensus.cc:2272] T ecc57d78878149a7b4ceb2a1ca320aab P b512283c6beb499997235d58fa0b2eae [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.631388 15687 tablet_replica.cc:333] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:14.631512 15687 raft_consensus.cc:2243] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 LEADER]: Raft consensus shutting down.
I20251023 20:18:14.631731 15687 raft_consensus.cc:2272] T 513ec58123084ca6903177be676f9bf9 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.632149 15687 tablet_replica.cc:333] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae: stopping tablet replica
I20251023 20:18:14.632282 15687 raft_consensus.cc:2243] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.632436 15687 raft_consensus.cc:2272] T 03ab8ce022984c379448ed4cd5b7ada6 P b512283c6beb499997235d58fa0b2eae [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.653679 15687 tablet_server.cc:195] TabletServer@127.15.81.195:0 shutdown complete.
I20251023 20:18:14.657630 15687 tablet_server.cc:178] TabletServer@127.15.81.196:0 shutting down...
I20251023 20:18:14.665750 15687 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 20:18:14.665987 15687 tablet_replica.cc:333] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8: stopping tablet replica
I20251023 20:18:14.666169 15687 raft_consensus.cc:2243] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.666332 15687 raft_consensus.cc:2272] T 4acf1d8c7934480fbf2873b9e9fa77d2 P 65552aac62fc40d49cefd020b8325ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.666771 15687 tablet_replica.cc:333] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8: stopping tablet replica
I20251023 20:18:14.666865 15687 raft_consensus.cc:2243] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.667025 15687 raft_consensus.cc:2272] T 513ec58123084ca6903177be676f9bf9 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.667359 15687 tablet_replica.cc:333] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8: stopping tablet replica
I20251023 20:18:14.667480 15687 raft_consensus.cc:2243] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 20:18:14.667635 15687 raft_consensus.cc:2272] T 03ab8ce022984c379448ed4cd5b7ada6 P 65552aac62fc40d49cefd020b8325ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.681047 15687 tablet_server.cc:195] TabletServer@127.15.81.196:0 shutdown complete.
I20251023 20:18:14.684036 15687 master.cc:561] Master@127.15.81.254:41703 shutting down...
I20251023 20:18:14.690471 15687 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 1 LEADER]: Raft consensus shutting down.
I20251023 20:18:14.690644 15687 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 20:18:14.690732 15687 tablet_replica.cc:333] T 00000000000000000000000000000000 P c8ea4045ecb74a829b54f4177fcb31cf: stopping tablet replica
I20251023 20:18:14.703469 15687 master.cc:583] Master@127.15.81.254:41703 shutdown complete.
[ OK ] AutoRebalancerTest.TestDeletedTables (8816 ms)
[----------] 14 tests from AutoRebalancerTest (187758 ms total)
[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (187758 ms total)
[ PASSED ] 13 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer
1 FAILED TEST
I20251023 20:18:14.716150 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251023 20:18:14.716192 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251023 20:18:14.716214 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/catalog_manager.cc:4712: suppressed but not reported on 13 messages since previous log ~5 seconds ago
I20251023 20:18:14.716262 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/catalog_manager.cc:5327: suppressed but not reported on 15 messages since previous log ~31 seconds ago
I20251023 20:18:14.716311 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 8 messages since previous log ~173 seconds ago
I20251023 20:18:14.716348 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 51 messages since previous log ~19 seconds ago
I20251023 20:18:14.716434 15687 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 29 messages since previous log ~8 seconds ago